2026-02-17T18:31:30.718 INFO:root:teuthology version: 0.0.1.dev31+g5f66ecfb3 2026-02-17T18:31:30.731 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T18:31:30.805 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_15:45:02-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/53767 branch: wip-yuri4-testing-2026-02-11-2126 description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/misc} email: null first_in_suite: false flavor: default job_id: '53767' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-17_15:45:02-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri4-testing-2026-02-11-2126 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 workunit: branch: wip-yuri4-testing-2026-02-11-2126 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 5319 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2026-02-11-2126 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 targets: trial068.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmqkqG8SgDsHgZJmmO47FgAziVnRyqhcgy2JzfgUilFH+6hhpKnBMrcxbHHChugWTxTXiba2ND8+PsdC21t838= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: 5f66ecfb34c0370410e78b3ee641753d19da653b timestamp: 2026-02-17_15:45:02 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T18:31:30.805 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa; will attempt to use it 2026-02-17T18:31:30.805 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks 2026-02-17T18:31:30.805 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T18:31:30.807 INFO:teuthology.task.internal:Checking packages... 2026-02-17T18:31:30.807 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'afefaa40503a4bf024e96f97dd7f852b75f00ad6' 2026-02-17T18:31:30.807 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T18:31:30.807 INFO:teuthology.packaging:ref: None 2026-02-17T18:31:30.807 INFO:teuthology.packaging:tag: None 2026-02-17T18:31:30.807 INFO:teuthology.packaging:branch: wip-yuri4-testing-2026-02-11-2126 2026-02-17T18:31:30.807 INFO:teuthology.packaging:sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-17T18:31:30.807 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri4-testing-2026-02-11-2126 2026-02-17T18:31:30.924 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5230.gafefaa40 2026-02-17T18:31:30.925 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T18:31:30.936 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T18:31:30.936 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T18:31:30.950 INFO:teuthology.task.internal:Saving configuration 2026-02-17T18:31:30.962 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T18:31:30.974 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T18:31:31.050 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial068.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-17_15:45:02-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/53767', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 18:26:32.596922', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmqkqG8SgDsHgZJmmO47FgAziVnRyqhcgy2JzfgUilFH+6hhpKnBMrcxbHHChugWTxTXiba2ND8+PsdC21t838='} 2026-02-17T18:31:31.050 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T18:31:31.062 INFO:teuthology.task.internal:roles: ubuntu@trial068.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T18:31:31.062 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T18:31:31.142 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f06e5008fe0>, signals=[15]) 2026-02-17T18:31:31.142 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T18:31:31.154 INFO:teuthology.task.internal:Opening connections... 2026-02-17T18:31:31.154 DEBUG:teuthology.task.internal:connecting to ubuntu@trial068.front.sepia.ceph.com 2026-02-17T18:31:31.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T18:31:31.215 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T18:31:31.225 DEBUG:teuthology.orchestra.run.trial068:> uname -m 2026-02-17T18:31:31.285 INFO:teuthology.orchestra.run.trial068.stdout:x86_64 2026-02-17T18:31:31.285 DEBUG:teuthology.orchestra.run.trial068:> cat /etc/os-release 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:NAME="CentOS Stream" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:VERSION="9" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:ID="centos" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:ID_LIKE="rhel fedora" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:VERSION_ID="9" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:PLATFORM_ID="platform:el9" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:ANSI_COLOR="0;31" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:LOGO="fedora-logo-icon" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:HOME_URL="https://centos.org/" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T18:31:31.341 INFO:teuthology.orchestra.run.trial068.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T18:31:31.342 INFO:teuthology.lock.ops:Updating trial068.front.sepia.ceph.com on lock server 2026-02-17T18:31:31.427 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T18:31:31.440 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T18:31:31.456 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T18:31:31.457 DEBUG:teuthology.orchestra.run.trial068:> test '!' -e /home/ubuntu/cephtest 2026-02-17T18:31:31.472 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T18:31:31.483 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T18:31:31.484 DEBUG:teuthology.orchestra.run.trial068:> test -z $(ls -A /var/lib/ceph) 2026-02-17T18:31:31.528 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T18:31:31.635 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T18:31:31.656 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-17T18:31:31.656 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-17T18:31:31.656 DEBUG:teuthology.orchestra.run.trial068:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T18:31:31.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T18:31:31.672 DEBUG:teuthology.orchestra.run.trial068:> uname -r 2026-02-17T18:31:31.727 INFO:teuthology.orchestra.run.trial068.stdout:5.14.0-645.el9.x86_64 2026-02-17T18:31:31.727 INFO:teuthology.task.kernel:Running kernel on trial068: 5.14.0-645.el9.x86_64 2026-02-17T18:31:31.728 DEBUG:teuthology.orchestra.run.trial068:> sudo yum install -y kernel 2026-02-17T18:31:35.939 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.9 MB 00:03 2026-02-17T18:31:38.978 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-17T18:31:46.596 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 7.8 MB 00:05 2026-02-17T18:31:47.736 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 20 kB 00:00 2026-02-17T18:31:48.327 INFO:teuthology.orchestra.run.trial068.stdout:Extra Packages for Enterprise Linux 38 MB/s | 20 MB 00:00 2026-02-17T18:31:51.534 INFO:teuthology.orchestra.run.trial068.stdout:lab-extras 932 kB/s | 50 kB 00:00 2026-02-17T18:31:52.546 INFO:teuthology.orchestra.run.trial068.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T18:31:52.567 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout: Package Architecture Version Repository Size 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T18:31:52.569 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:31:52.570 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:31:52.570 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:31:52.570 INFO:teuthology.orchestra.run.trial068.stdout:Install 4 Packages 2026-02-17T18:31:52.570 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:31:52.570 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 88 M 2026-02-17T18:31:52.570 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 135 M 2026-02-17T18:31:52.570 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:31:53.286 INFO:teuthology.orchestra.run.trial068.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2026-02-17T18:31:53.829 INFO:teuthology.orchestra.run.trial068.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 14 MB/s | 17 MB 00:01 2026-02-17T18:31:54.335 INFO:teuthology.orchestra.run.trial068.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 29 MB/s | 31 MB 00:01 2026-02-17T18:31:54.486 INFO:teuthology.orchestra.run.trial068.stdout:(4/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 21 MB/s | 40 MB 00:01 2026-02-17T18:31:54.486 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:31:54.486 INFO:teuthology.orchestra.run.trial068.stdout:Total 46 MB/s | 88 MB 00:01 2026-02-17T18:31:54.677 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:31:54.880 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:31:54.880 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:31:55.229 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:31:55.230 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:31:56.316 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:31:56.597 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T18:31:56.616 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T18:31:58.399 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T18:31:58.428 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T18:32:01.450 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T18:32:01.456 INFO:teuthology.orchestra.run.trial068.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T18:32:04.489 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T18:32:34.834 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T18:32:34.855 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T18:32:34.919 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T18:32:34.919 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T18:32:34.920 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T18:32:34.920 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T18:32:34.971 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:32:34.972 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:32:35.030 DEBUG:teuthology.orchestra.run.trial068:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T18:32:35.055 INFO:teuthology.orchestra.run.trial068.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T18:32:35.056 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T18:32:35.056 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T18:32:35.057 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T18:32:35.057 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T18:32:35.057 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T18:32:35.057 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial068.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T18:32:35.057 DEBUG:teuthology.orchestra.run.trial068:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T18:32:35.119 INFO:teuthology.orchestra.run.trial068.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T18:32:35.120 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T18:32:35.121 DEBUG:teuthology.orchestra.run.trial068:> sudo rpm -qi grub2-tools 2026-02-17T18:32:35.188 INFO:teuthology.orchestra.run.trial068.stdout:Name : grub2-tools 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Epoch : 1 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Version : 2.06 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Release : 118.el9 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Architecture: x86_64 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Group : Unspecified 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Size : 8393909 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:License : GPLv3+ 2026-02-17T18:32:35.189 INFO:teuthology.orchestra.run.trial068.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:Packager : builder@centos.org 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:Vendor : CentOS 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:Summary : Support tools for GRUB. 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:Description : 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout:hardware devices. 2026-02-17T18:32:35.190 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:32:35.191 INFO:teuthology.orchestra.run.trial068.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T18:32:35.192 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T18:32:35.192 INFO:teuthology.task.kernel:Updating grub on trial068 to boot 5.14.0-681.el9.x86_64 2026-02-17T18:32:35.192 DEBUG:teuthology.orchestra.run.trial068:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T18:32:35.255 INFO:teuthology.orchestra.run.trial068.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T18:32:35.257 DEBUG:teuthology.orchestra.run.trial068:> sudo command -v grubby && echo yes || echo no 2026-02-17T18:32:35.321 INFO:teuthology.orchestra.run.trial068.stdout:/sbin/grubby 2026-02-17T18:32:35.323 INFO:teuthology.orchestra.run.trial068.stdout:yes 2026-02-17T18:32:35.323 DEBUG:teuthology.orchestra.run.trial068:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T18:32:35.387 INFO:teuthology.orchestra.run.trial068.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T18:32:35.388 DEBUG:teuthology.orchestra.run.trial068:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T18:32:35.461 DEBUG:teuthology.orchestra.run.trial068:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T18:32:35.673 INFO:teuthology.orchestra.run.trial068.stderr:Generating grub configuration file ... 2026-02-17T18:32:35.736 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T18:32:35.742 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T18:32:35.992 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T18:32:35.998 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T18:32:36.029 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T18:32:36.036 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T18:32:36.059 INFO:teuthology.orchestra.run.trial068.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T18:32:36.065 INFO:teuthology.orchestra.run.trial068.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T18:32:36.148 INFO:teuthology.orchestra.run.trial068.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T18:32:36.156 INFO:teuthology.orchestra.run.trial068.stderr:done 2026-02-17T18:32:36.157 DEBUG:teuthology.orchestra.run.trial068:> test -d /sys/firmware/efi 2026-02-17T18:32:36.172 DEBUG:teuthology.orchestra.run.trial068:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T18:32:36.237 INFO:teuthology.orchestra.run.trial068.stdout:/boot/efi/EFI/BOOT 2026-02-17T18:32:36.237 INFO:teuthology.orchestra.run.trial068.stdout:/boot/efi/EFI/centos 2026-02-17T18:32:36.238 DEBUG:teuthology.orchestra.run.trial068:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T18:32:36.304 INFO:teuthology.orchestra.run.trial068.stdout:no 2026-02-17T18:32:36.304 DEBUG:teuthology.orchestra.run.trial068:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T18:32:36.367 INFO:teuthology.orchestra.run.trial068.stdout:yes 2026-02-17T18:32:36.368 DEBUG:teuthology.orchestra.run.trial068:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T18:32:36.434 DEBUG:teuthology.orchestra.run.trial068:> sudo shutdown -r now 2026-02-17T18:33:06.505 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T18:33:06.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-17T18:33:06.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T18:34:06.566 DEBUG:teuthology.orchestra.remote:timed out 2026-02-17T18:34:15.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-17T18:34:15.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T18:34:15.664 DEBUG:teuthology.orchestra.run.trial068:> true 2026-02-17T18:34:15.819 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-17T18:34:15.819 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-681.el9.x86_64"... 2026-02-17T18:34:15.819 DEBUG:teuthology.orchestra.run.trial068:> uname -r 2026-02-17T18:34:15.875 INFO:teuthology.orchestra.run.trial068.stdout:5.14.0-681.el9.x86_64 2026-02-17T18:34:15.876 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T18:34:15.876 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T18:34:15.876 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T18:34:16.877 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T18:34:16.877 DEBUG:teuthology.orchestra.run.trial068:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T18:34:16.908 INFO:teuthology.orchestra.run.trial068.stdout:ttyS1 2026-02-17T18:34:16.932 DEBUG:teuthology.parallel:result is None 2026-02-17T18:34:16.932 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T18:34:16.948 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T18:34:16.949 DEBUG:teuthology.orchestra.run.trial068:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T18:34:16.989 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T18:34:17.000 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T18:34:17.013 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T18:34:17.013 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T18:34:17.046 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T18:34:17.057 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T18:34:17.057 DEBUG:teuthology.orchestra.run.trial068:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T18:34:17.109 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:34:17.117 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:34:17.119 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T18:34:17.130 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T18:34:17.130 DEBUG:teuthology.orchestra.run.trial068:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T18:34:17.183 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T18:34:17.198 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T18:34:17.199 DEBUG:teuthology.orchestra.run.trial068:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T18:34:17.238 DEBUG:teuthology.orchestra.run.trial068:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T18:34:17.372 DEBUG:teuthology.orchestra.run.trial068:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T18:34:17.397 DEBUG:teuthology.orchestra.run.trial068:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T18:34:17.531 DEBUG:teuthology.orchestra.run.trial068:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T18:34:17.555 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-17T18:34:17.556 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T18:34:17.619 DEBUG:teuthology.orchestra.run.trial068:> sudo service rsyslog restart 2026-02-17T18:34:17.686 INFO:teuthology.orchestra.run.trial068.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T18:34:17.890 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T18:34:17.903 INFO:teuthology.task.internal:Starting timer... 2026-02-17T18:34:17.904 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T18:34:17.925 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T18:34:17.944 DEBUG:teuthology.orchestra.run.trial068:> sudo service auditd rotate 2026-02-17T18:34:17.973 INFO:teuthology.orchestra.run.trial068.stdout:Rotating logs: 2026-02-17T18:34:17.975 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T18:34:17.975 DEBUG:teuthology.orchestra.run.trial068:> /usr/sbin/getenforce 2026-02-17T18:34:18.031 INFO:teuthology.orchestra.run.trial068.stdout:Permissive 2026-02-17T18:34:18.032 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial068.front.sepia.ceph.com': 'permissive'} 2026-02-17T18:34:18.032 DEBUG:teuthology.orchestra.run.trial068:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-17T18:34:18.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T18:34:18.096 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T18:34:18.097 DEBUG:teuthology.orchestra.run.trial068:> sudo /usr/sbin/setenforce permissive 2026-02-17T18:34:18.161 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T18:34:18.178 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-17T18:34:18.180 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-17T18:34:18.285 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T18:34:18.293 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T18:34:18.294 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial068.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T18:35:27.683 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial068.front.sepia.ceph.com')] 2026-02-17T18:35:27.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-17T18:35:27.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial068.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T18:35:27.773 DEBUG:teuthology.orchestra.run.trial068:> true 2026-02-17T18:35:27.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial068.front.sepia.ceph.com' 2026-02-17T18:35:27.850 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T18:35:27.864 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T18:35:27.864 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T18:35:27.864 DEBUG:teuthology.orchestra.run.trial068:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T18:35:27.917 INFO:teuthology.orchestra.run.trial068.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T18:35:27.929 INFO:teuthology.orchestra.run.trial068.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T18:35:27.949 INFO:teuthology.orchestra.run.trial068.stderr:sudo: ntpd: command not found 2026-02-17T18:35:27.959 INFO:teuthology.orchestra.run.trial068.stdout:506 Cannot talk to daemon 2026-02-17T18:35:27.971 INFO:teuthology.orchestra.run.trial068.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T18:35:27.983 INFO:teuthology.orchestra.run.trial068.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T18:35:28.030 INFO:teuthology.orchestra.run.trial068.stderr:bash: line 1: ntpq: command not found 2026-02-17T18:35:28.059 INFO:teuthology.orchestra.run.trial068.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T18:35:28.059 INFO:teuthology.orchestra.run.trial068.stdout:=============================================================================== 2026-02-17T18:35:28.059 INFO:teuthology.orchestra.run.trial068.stdout:^? ntp.maxhost.io 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T18:35:28.059 INFO:teuthology.orchestra.run.trial068.stdout:^? ntp.netlinkify.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T18:35:28.059 INFO:teuthology.orchestra.run.trial068.stdout:^? anaheim.capsaic.in 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T18:35:28.059 INFO:teuthology.orchestra.run.trial068.stdout:^? 143-42-229-154.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T18:35:28.061 INFO:teuthology.run_tasks:Running task install... 2026-02-17T18:35:28.075 DEBUG:teuthology.task.install:project ceph 2026-02-17T18:35:28.075 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'}} 2026-02-17T18:35:28.075 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'} 2026-02-17T18:35:28.075 INFO:teuthology.task.install:Using flavor: default 2026-02-17T18:35:28.085 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T18:35:28.085 INFO:teuthology.task.install:extra packages: [] 2026-02-17T18:35:28.085 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6', 'tag': None, 'wait_for_package': False} 2026-02-17T18:35:28.086 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-17T18:35:28.250 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/ 2026-02-17T18:35:28.250 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5230.gafefaa40 2026-02-17T18:35:28.557 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T18:35:28.557 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-17T18:35:28.558 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T18:35:28.581 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-17T18:35:28.581 DEBUG:teuthology.orchestra.run.trial068:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/afefaa40503a4bf024e96f97dd7f852b75f00ad6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T18:35:28.645 DEBUG:teuthology.orchestra.run.trial068:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T18:35:28.718 DEBUG:teuthology.orchestra.run.trial068:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T18:35:28.781 INFO:teuthology.orchestra.run.trial068.stdout:check_obsoletes = 1 2026-02-17T18:35:28.782 DEBUG:teuthology.orchestra.run.trial068:> sudo yum clean all 2026-02-17T18:35:28.932 INFO:teuthology.orchestra.run.trial068.stdout:41 files removed 2026-02-17T18:35:28.946 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-radosgw 2026-02-17T18:35:29.641 INFO:teuthology.orchestra.run.trial068.stdout:ceph packages for x86_64 167 kB/s | 91 kB 00:00 2026-02-17T18:35:30.223 INFO:teuthology.orchestra.run.trial068.stdout:ceph noarch packages 42 kB/s | 24 kB 00:00 2026-02-17T18:35:30.695 INFO:teuthology.orchestra.run.trial068.stdout:ceph source packages 4.6 kB/s | 2.1 kB 00:00 2026-02-17T18:35:31.502 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-17T18:35:36.028 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 27 MB 00:03 2026-02-17T18:35:39.658 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 7.8 MB 00:01 2026-02-17T18:35:40.499 INFO:teuthology.orchestra.run.trial068.stdout:CentOS Stream 9 - Extras packages 192 kB/s | 20 kB 00:00 2026-02-17T18:35:41.320 INFO:teuthology.orchestra.run.trial068.stdout:Extra Packages for Enterprise Linux 27 MB/s | 20 MB 00:00 2026-02-17T18:35:44.567 INFO:teuthology.orchestra.run.trial068.stdout:lab-extras 1.3 MB/s | 50 kB 00:00 2026-02-17T18:35:45.784 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: ceph-radosgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 13 M 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout:Upgrading: 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: librados2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 3.5 M 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: librbd1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.8 M 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: ceph-base x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.2 M 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 24 M 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: ceph-selinux x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 909 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: libradosstriper1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 248 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: librgw2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 M 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-argparse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 45 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 185 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: python3-cephfs x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 203 k 2026-02-17T18:35:45.786 INFO:teuthology.orchestra.run.trial068.stdout: python3-rados x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 324 k 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: python3-rbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 305 k 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: python3-rgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 100 k 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout:Installing weak dependencies: 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout:Install 32 Packages 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout:Upgrade 2 Packages 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 67 M 2026-02-17T18:35:45.787 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:35:48.586 INFO:teuthology.orchestra.run.trial068.stdout:(1/34): ceph-base-20.3.0-5230.gafefaa40.el9.x86 2.3 MB/s | 5.2 MB 00:02 2026-02-17T18:35:48.645 INFO:teuthology.orchestra.run.trial068.stdout:(2/34): ceph-selinux-20.3.0-5230.gafefaa40.el9. 423 kB/s | 25 kB 00:00 2026-02-17T18:35:48.773 INFO:teuthology.orchestra.run.trial068.stdout:(3/34): libcephfs2-20.3.0-5230.gafefaa40.el9.x8 6.9 MB/s | 909 kB 00:00 2026-02-17T18:35:48.816 INFO:teuthology.orchestra.run.trial068.stdout:(4/34): libradosstriper1-20.3.0-5230.gafefaa40. 5.7 MB/s | 248 kB 00:00 2026-02-17T18:35:49.547 INFO:teuthology.orchestra.run.trial068.stdout:(5/34): ceph-radosgw-20.3.0-5230.gafefaa40.el9. 4.1 MB/s | 13 MB 00:03 2026-02-17T18:35:49.566 INFO:teuthology.orchestra.run.trial068.stdout:(6/34): librgw2-20.3.0-5230.gafefaa40.el9.x86_6 8.6 MB/s | 6.5 MB 00:00 2026-02-17T18:35:49.566 INFO:teuthology.orchestra.run.trial068.stdout:(7/34): python3-ceph-argparse-20.3.0-5230.gafef 2.3 MB/s | 45 kB 00:00 2026-02-17T18:35:49.588 INFO:teuthology.orchestra.run.trial068.stdout:(8/34): python3-ceph-common-20.3.0-5230.gafefaa 8.1 MB/s | 185 kB 00:00 2026-02-17T18:35:49.589 INFO:teuthology.orchestra.run.trial068.stdout:(9/34): python3-cephfs-20.3.0-5230.gafefaa40.el 8.8 MB/s | 203 kB 00:00 2026-02-17T18:35:49.616 INFO:teuthology.orchestra.run.trial068.stdout:(10/34): python3-rados-20.3.0-5230.gafefaa40.el 12 MB/s | 324 kB 00:00 2026-02-17T18:35:49.618 INFO:teuthology.orchestra.run.trial068.stdout:(11/34): python3-rbd-20.3.0-5230.gafefaa40.el9. 10 MB/s | 305 kB 00:00 2026-02-17T18:35:49.629 INFO:teuthology.orchestra.run.trial068.stdout:(12/34): python3-rgw-20.3.0-5230.gafefaa40.el9. 7.3 MB/s | 100 kB 00:00 2026-02-17T18:35:49.672 INFO:teuthology.orchestra.run.trial068.stdout:(13/34): mailcap-2.1.49-5.el9.noarch.rpm 618 kB/s | 33 kB 00:00 2026-02-17T18:35:49.696 INFO:teuthology.orchestra.run.trial068.stdout:(14/34): pciutils-3.7.0-7.el9.x86_64.rpm 1.4 MB/s | 93 kB 00:00 2026-02-17T18:35:50.113 INFO:teuthology.orchestra.run.trial068.stdout:(15/34): ceph-common-20.3.0-5230.gafefaa40.el9. 6.3 MB/s | 24 MB 00:03 2026-02-17T18:35:50.255 INFO:teuthology.orchestra.run.trial068.stdout:(16/34): boost-program-options-1.75.0-13.el9.x8 179 kB/s | 104 kB 00:00 2026-02-17T18:35:50.277 INFO:teuthology.orchestra.run.trial068.stdout:(17/34): libnbd-1.20.3-4.el9.x86_64.rpm 282 kB/s | 164 kB 00:00 2026-02-17T18:35:50.341 INFO:teuthology.orchestra.run.trial068.stdout:(18/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 526 kB/s | 45 kB 00:00 2026-02-17T18:35:50.528 INFO:teuthology.orchestra.run.trial068.stdout:(19/34): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2026-02-17T18:35:50.615 INFO:teuthology.orchestra.run.trial068.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 2.7 MB/s | 240 kB 00:00 2026-02-17T18:35:50.697 INFO:teuthology.orchestra.run.trial068.stdout:(21/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 274 kB/s | 160 kB 00:00 2026-02-17T18:35:50.702 INFO:teuthology.orchestra.run.trial068.stdout:(22/34): qatlib-service-25.08.0-2.el9.x86_64.rp 427 kB/s | 37 kB 00:00 2026-02-17T18:35:50.751 INFO:teuthology.orchestra.run.trial068.stdout:(23/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 713 kB/s | 292 kB 00:00 2026-02-17T18:35:50.783 INFO:teuthology.orchestra.run.trial068.stdout:(24/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 767 kB/s | 66 kB 00:00 2026-02-17T18:35:50.951 INFO:teuthology.orchestra.run.trial068.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 147 kB/s | 25 kB 00:00 2026-02-17T18:35:50.997 INFO:teuthology.orchestra.run.trial068.stdout:(26/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.0 MB/s | 308 kB 00:00 2026-02-17T18:35:51.035 INFO:teuthology.orchestra.run.trial068.stdout:(27/34): liboath-2.6.12-1.el9.x86_64.rpm 585 kB/s | 49 kB 00:00 2026-02-17T18:35:51.039 INFO:teuthology.orchestra.run.trial068.stdout:(28/34): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2026-02-17T18:35:51.092 INFO:teuthology.orchestra.run.trial068.stdout:(29/34): re2-20211101-20.el9.x86_64.rpm 3.5 MB/s | 191 kB 00:00 2026-02-17T18:35:51.205 INFO:teuthology.orchestra.run.trial068.stdout:(30/34): libarrow-9.0.0-15.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2026-02-17T18:35:51.219 INFO:teuthology.orchestra.run.trial068.stdout:(31/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2026-02-17T18:35:51.228 INFO:teuthology.orchestra.run.trial068.stdout:(32/34): thrift-0.15.0-4.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2026-02-17T18:35:52.030 INFO:teuthology.orchestra.run.trial068.stdout:(33/34): librbd1-20.3.0-5230.gafefaa40.el9.x86_ 3.5 MB/s | 2.8 MB 00:00 2026-02-17T18:35:52.189 INFO:teuthology.orchestra.run.trial068.stdout:(34/34): librados2-20.3.0-5230.gafefaa40.el9.x8 3.5 MB/s | 3.5 MB 00:00 2026-02-17T18:35:52.195 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:35:52.195 INFO:teuthology.orchestra.run.trial068.stdout:Total 10 MB/s | 67 MB 00:06 2026-02-17T18:35:52.286 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:35:52.304 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:35:52.304 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:35:52.566 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:35:52.566 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:35:52.867 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:35:53.020 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-17T18:35:53.025 INFO:teuthology.orchestra.run.trial068.stdout: Upgrading : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-17T18:35:53.055 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-17T18:35:53.094 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-17T18:35:53.095 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-17T18:35:53.107 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-17T18:35:53.115 INFO:teuthology.orchestra.run.trial068.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-17T18:35:53.117 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-17T18:35:53.120 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-17T18:35:53.124 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-17T18:35:53.135 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 9/36 2026-02-17T18:35:53.147 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-17T18:35:53.148 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-17T18:35:53.172 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-17T18:35:53.176 INFO:teuthology.orchestra.run.trial068.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-17T18:35:53.211 INFO:teuthology.orchestra.run.trial068.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-17T18:35:53.214 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-17T18:35:53.222 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-17T18:35:53.224 INFO:teuthology.orchestra.run.trial068.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-17T18:35:53.229 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-17T18:35:53.350 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-17T18:35:53.351 INFO:teuthology.orchestra.run.trial068.stdout: Upgrading : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-17T18:35:53.377 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-17T18:35:53.379 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 20/36 2026-02-17T18:35:53.385 INFO:teuthology.orchestra.run.trial068.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-17T18:35:53.667 INFO:teuthology.orchestra.run.trial068.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-17T18:35:53.668 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-17T18:35:53.684 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-17T18:35:53.688 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 24/36 2026-02-17T18:35:53.694 INFO:teuthology.orchestra.run.trial068.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-17T18:35:53.696 INFO:teuthology.orchestra.run.trial068.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-17T18:35:53.697 INFO:teuthology.orchestra.run.trial068.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-17T18:35:53.715 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-17T18:35:53.717 INFO:teuthology.orchestra.run.trial068.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-17T18:35:53.751 INFO:teuthology.orchestra.run.trial068.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-17T18:35:53.752 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 30/36 2026-02-17T18:35:54.844 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-17T18:35:54.847 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-17T18:35:55.111 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-17T18:35:55.114 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-17T18:35:55.154 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-17T18:35:55.154 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T18:35:55.154 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T18:35:55.154 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:35:55.164 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /sys 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /proc 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /mnt 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /var/tmp 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /home 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /root 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout:skipping the directory /tmp 2026-02-17T18:35:59.678 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:00.253 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-17T18:36:00.275 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-17T18:36:00.275 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:00.275 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T18:36:00.275 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T18:36:00.275 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T18:36:00.275 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:00.277 INFO:teuthology.orchestra.run.trial068.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-17T18:36:00.288 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-17T18:36:00.288 INFO:teuthology.orchestra.run.trial068.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-17T18:36:00.343 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-17T18:36:00.343 INFO:teuthology.orchestra.run.trial068.stdout:Creating group 'qat' with GID 992. 2026-02-17T18:36:00.343 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:00.640 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 1/36 2026-02-17T18:36:00.640 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-17T18:36:00.640 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-17T18:36:00.640 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 5/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 6/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 7/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 8/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 9/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 11/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 12/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 13/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-17T18:36:00.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-17T18:36:00.642 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-17T18:36:00.643 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-17T18:36:00.643 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 35/36 2026-02-17T18:36:00.696 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-17T18:36:00.696 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:00.696 INFO:teuthology.orchestra.run.trial068.stdout:Upgraded: 2026-02-17T18:36:00.696 INFO:teuthology.orchestra.run.trial068.stdout: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.696 INFO:teuthology.orchestra.run.trial068.stdout: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T18:36:00.697 INFO:teuthology.orchestra.run.trial068.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.698 INFO:teuthology.orchestra.run.trial068.stdout: python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:00.699 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:00.738 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-test 2026-02-17T18:36:01.047 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:17 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:01.128 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: ceph-test x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 78 M 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: libcephsqlite x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 159 k 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T18:36:01.129 INFO:teuthology.orchestra.run.trial068.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout:Install 10 Packages 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 82 M 2026-02-17T18:36:01.130 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 339 M 2026-02-17T18:36:01.131 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:01.587 INFO:teuthology.orchestra.run.trial068.stdout:(1/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 14 MB/s | 2.1 MB 00:00 2026-02-17T18:36:01.609 INFO:teuthology.orchestra.run.trial068.stdout:(2/10): libatomic-11.5.0-14.el9.x86_64.rpm 1.0 MB/s | 23 kB 00:00 2026-02-17T18:36:01.632 INFO:teuthology.orchestra.run.trial068.stdout:(3/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 4.1 MB/s | 94 kB 00:00 2026-02-17T18:36:01.638 INFO:teuthology.orchestra.run.trial068.stdout:(4/10): libcephsqlite-20.3.0-5230.gafefaa40.el9 786 kB/s | 159 kB 00:00 2026-02-17T18:36:01.683 INFO:teuthology.orchestra.run.trial068.stdout:(5/10): libxslt-1.1.34-12.el9.x86_64.rpm 4.4 MB/s | 233 kB 00:00 2026-02-17T18:36:01.700 INFO:teuthology.orchestra.run.trial068.stdout:(6/10): protobuf-3.14.0-17.el9.x86_64.rpm 16 MB/s | 1.0 MB 00:00 2026-02-17T18:36:01.702 INFO:teuthology.orchestra.run.trial068.stdout:(7/10): socat-1.7.4.1-8.el9.x86_64.rpm 16 MB/s | 303 kB 00:00 2026-02-17T18:36:01.709 INFO:teuthology.orchestra.run.trial068.stdout:(8/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 6.9 MB/s | 64 kB 00:00 2026-02-17T18:36:01.917 INFO:teuthology.orchestra.run.trial068.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 567 kB/s | 122 kB 00:00 2026-02-17T18:36:03.806 INFO:teuthology.orchestra.run.trial068.stdout:(10/10): ceph-test-20.3.0-5230.gafefaa40.el9.x8 33 MB/s | 78 MB 00:02 2026-02-17T18:36:03.808 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:03.808 INFO:teuthology.orchestra.run.trial068.stdout:Total 31 MB/s | 82 MB 00:02 2026-02-17T18:36:03.872 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:03.882 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:03.883 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:04.166 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:04.166 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:04.469 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:04.477 INFO:teuthology.orchestra.run.trial068.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-17T18:36:04.486 INFO:teuthology.orchestra.run.trial068.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-17T18:36:04.491 INFO:teuthology.orchestra.run.trial068.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-17T18:36:04.495 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-17T18:36:04.500 INFO:teuthology.orchestra.run.trial068.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-17T18:36:04.502 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-17T18:36:04.512 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-17T18:36:04.520 INFO:teuthology.orchestra.run.trial068.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-17T18:36:04.521 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-17T18:36:07.960 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-17T18:36:07.978 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-17T18:36:08.282 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-17T18:36:08.282 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 1/10 2026-02-17T18:36:08.282 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2/10 2026-02-17T18:36:08.283 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-17T18:36:08.283 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-17T18:36:08.283 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-17T18:36:08.283 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-17T18:36:08.283 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-17T18:36:08.283 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-17T18:36:08.283 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T18:36:08.335 INFO:teuthology.orchestra.run.trial068.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T18:36:08.336 INFO:teuthology.orchestra.run.trial068.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T18:36:08.336 INFO:teuthology.orchestra.run.trial068.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-17T18:36:08.336 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:08.336 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:08.372 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph 2026-02-17T18:36:08.683 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:24 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:08.766 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mds x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.4 M 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 936 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 297 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mon x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.0 M 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 166 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-classic x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 16 M 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-crimson x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 19 M 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T18:36:08.767 INFO:teuthology.orchestra.run.trial068.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-collections 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout:Install 42 Packages 2026-02-17T18:36:08.768 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:08.769 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 48 M 2026-02-17T18:36:08.769 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 167 M 2026-02-17T18:36:08.769 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:09.413 INFO:teuthology.orchestra.run.trial068.stdout:(1/42): ceph-20.3.0-5230.gafefaa40.el9.x86_64.r 67 kB/s | 6.5 kB 00:00 2026-02-17T18:36:09.595 INFO:teuthology.orchestra.run.trial068.stdout:(2/42): ceph-mgr-20.3.0-5230.gafefaa40.el9.x86_ 3.3 MB/s | 936 kB 00:00 2026-02-17T18:36:09.619 INFO:teuthology.orchestra.run.trial068.stdout:(3/42): ceph-osd-20.3.0-5230.gafefaa40.el9.x86_ 6.8 MB/s | 166 kB 00:00 2026-02-17T18:36:09.782 INFO:teuthology.orchestra.run.trial068.stdout:(4/42): ceph-mds-20.3.0-5230.gafefaa40.el9.x86_ 5.1 MB/s | 2.4 MB 00:00 2026-02-17T18:36:10.119 INFO:teuthology.orchestra.run.trial068.stdout:(5/42): ceph-mon-20.3.0-5230.gafefaa40.el9.x86_ 7.0 MB/s | 5.0 MB 00:00 2026-02-17T18:36:10.167 INFO:teuthology.orchestra.run.trial068.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5230.gafef 6.1 MB/s | 297 kB 00:00 2026-02-17T18:36:10.199 INFO:teuthology.orchestra.run.trial068.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.2 MB/s | 40 kB 00:00 2026-02-17T18:36:10.215 INFO:teuthology.orchestra.run.trial068.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 4.5 MB/s | 72 kB 00:00 2026-02-17T18:36:10.234 INFO:teuthology.orchestra.run.trial068.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 13 MB/s | 253 kB 00:00 2026-02-17T18:36:10.270 INFO:teuthology.orchestra.run.trial068.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 35 MB/s | 1.2 MB 00:00 2026-02-17T18:36:10.280 INFO:teuthology.orchestra.run.trial068.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 10 MB/s | 106 kB 00:00 2026-02-17T18:36:10.290 INFO:teuthology.orchestra.run.trial068.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 13 MB/s | 135 kB 00:00 2026-02-17T18:36:10.301 INFO:teuthology.orchestra.run.trial068.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 14 MB/s | 150 kB 00:00 2026-02-17T18:36:10.312 INFO:teuthology.orchestra.run.trial068.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 11 MB/s | 126 kB 00:00 2026-02-17T18:36:10.323 INFO:teuthology.orchestra.run.trial068.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 19 MB/s | 218 kB 00:00 2026-02-17T18:36:10.335 INFO:teuthology.orchestra.run.trial068.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 16 MB/s | 182 kB 00:00 2026-02-17T18:36:10.345 INFO:teuthology.orchestra.run.trial068.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 25 MB/s | 266 kB 00:00 2026-02-17T18:36:10.505 INFO:teuthology.orchestra.run.trial068.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 1.5 MB/s | 246 kB 00:00 2026-02-17T18:36:10.542 INFO:teuthology.orchestra.run.trial068.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 5.0 MB/s | 188 kB 00:00 2026-02-17T18:36:10.578 INFO:teuthology.orchestra.run.trial068.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 4.8 MB/s | 177 kB 00:00 2026-02-17T18:36:10.608 INFO:teuthology.orchestra.run.trial068.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 2.6 MB/s | 77 kB 00:00 2026-02-17T18:36:10.635 INFO:teuthology.orchestra.run.trial068.stdout:(22/42): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 42 kB 00:00 2026-02-17T18:36:10.684 INFO:teuthology.orchestra.run.trial068.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 465 kB/s | 22 kB 00:00 2026-02-17T18:36:10.889 INFO:teuthology.orchestra.run.trial068.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 739 kB/s | 151 kB 00:00 2026-02-17T18:36:10.921 INFO:teuthology.orchestra.run.trial068.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 906 kB/s | 29 kB 00:00 2026-02-17T18:36:10.967 INFO:teuthology.orchestra.run.trial068.stdout:(26/42): ceph-osd-classic-20.3.0-5230.gafefaa40 12 MB/s | 16 MB 00:01 2026-02-17T18:36:10.967 INFO:teuthology.orchestra.run.trial068.stdout:(27/42): python3-backports-tarfile-1.2.0-1.el9. 1.3 MB/s | 60 kB 00:00 2026-02-17T18:36:11.010 INFO:teuthology.orchestra.run.trial068.stdout:(28/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2026-02-17T18:36:11.043 INFO:teuthology.orchestra.run.trial068.stdout:(29/42): python3-cheroot-10.0.1-4.el9.noarch.rp 2.2 MB/s | 173 kB 00:00 2026-02-17T18:36:11.073 INFO:teuthology.orchestra.run.trial068.stdout:(30/42): python3-cherrypy-18.6.1-2.el9.noarch.r 5.6 MB/s | 358 kB 00:00 2026-02-17T18:36:11.074 INFO:teuthology.orchestra.run.trial068.stdout:(31/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 342 kB/s | 11 kB 00:00 2026-02-17T18:36:11.105 INFO:teuthology.orchestra.run.trial068.stdout:(32/42): python3-jaraco-classes-3.2.1-5.el9.noa 558 kB/s | 18 kB 00:00 2026-02-17T18:36:11.119 INFO:teuthology.orchestra.run.trial068.stdout:(33/42): python3-jaraco-collections-3.0.0-8.el9 517 kB/s | 23 kB 00:00 2026-02-17T18:36:11.137 INFO:teuthology.orchestra.run.trial068.stdout:(34/42): python3-jaraco-context-6.0.1-3.el9.noa 607 kB/s | 20 kB 00:00 2026-02-17T18:36:11.174 INFO:teuthology.orchestra.run.trial068.stdout:(35/42): ceph-osd-crimson-20.3.0-5230.gafefaa40 14 MB/s | 19 MB 00:01 2026-02-17T18:36:11.175 INFO:teuthology.orchestra.run.trial068.stdout:(36/42): python3-jaraco-functools-3.5.0-2.el9.n 347 kB/s | 19 kB 00:00 2026-02-17T18:36:11.175 INFO:teuthology.orchestra.run.trial068.stdout:(37/42): python3-jaraco-text-4.0.0-2.el9.noarch 699 kB/s | 26 kB 00:00 2026-02-17T18:36:11.209 INFO:teuthology.orchestra.run.trial068.stdout:(38/42): python3-more-itertools-8.12.0-2.el9.no 2.2 MB/s | 79 kB 00:00 2026-02-17T18:36:11.210 INFO:teuthology.orchestra.run.trial068.stdout:(39/42): python3-portend-3.1.0-2.el9.noarch.rpm 465 kB/s | 16 kB 00:00 2026-02-17T18:36:11.215 INFO:teuthology.orchestra.run.trial068.stdout:(40/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.2 MB/s | 90 kB 00:00 2026-02-17T18:36:11.242 INFO:teuthology.orchestra.run.trial068.stdout:(41/42): python3-tempora-5.0.0-2.el9.noarch.rpm 1.1 MB/s | 36 kB 00:00 2026-02-17T18:36:11.245 INFO:teuthology.orchestra.run.trial068.stdout:(42/42): python3-zc-lockfile-2.0-10.el9.noarch. 585 kB/s | 20 kB 00:00 2026-02-17T18:36:11.248 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:11.248 INFO:teuthology.orchestra.run.trial068.stdout:Total 19 MB/s | 48 MB 00:02 2026-02-17T18:36:11.390 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:11.409 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:11.409 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:11.604 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:11.605 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:11.821 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:11.823 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-17T18:36:11.828 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-17T18:36:11.833 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-17T18:36:11.834 INFO:teuthology.orchestra.run.trial068.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-17T18:36:11.840 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-17T18:36:11.864 INFO:teuthology.orchestra.run.trial068.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-17T18:36:11.867 INFO:teuthology.orchestra.run.trial068.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-17T18:36:11.872 INFO:teuthology.orchestra.run.trial068.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-17T18:36:11.876 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-17T18:36:11.883 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-17T18:36:11.886 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-17T18:36:11.892 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-17T18:36:11.897 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-17T18:36:11.901 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-17T18:36:11.906 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-17T18:36:11.910 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-17T18:36:11.917 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-17T18:36:11.920 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-17T18:36:11.943 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-17T18:36:11.955 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-17T18:36:11.960 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-17T18:36:11.970 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-17T18:36:11.976 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-17T18:36:11.990 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-17T18:36:12.005 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-17T18:36:12.079 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-17T18:36:12.088 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-17T18:36:12.111 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-17T18:36:12.140 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-17T18:36:12.145 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-17T18:36:12.196 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-17T18:36:12.240 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 32/42 2026-02-17T18:36:12.240 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-17T18:36:12.264 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-17T18:36:12.264 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:12.264 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T18:36:12.264 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T18:36:12.264 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T18:36:12.264 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:12.268 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-17T18:36:12.286 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-17T18:36:12.286 INFO:teuthology.orchestra.run.trial068.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T18:36:12.286 INFO:teuthology.orchestra.run.trial068.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T18:36:12.286 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:12.294 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-17T18:36:12.318 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-17T18:36:12.318 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T18:36:12.318 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:12.962 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-17T18:36:13.734 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 37/42 2026-02-17T18:36:13.744 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 38/42 2026-02-17T18:36:13.745 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-17T18:36:13.769 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-17T18:36:13.769 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:13.769 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T18:36:13.769 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T18:36:13.769 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T18:36:13.769 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:13.983 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-17T18:36:14.004 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-17T18:36:14.004 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:14.004 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T18:36:14.004 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T18:36:14.004 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T18:36:14.004 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:14.100 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-17T18:36:14.120 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-17T18:36:14.121 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:14.121 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T18:36:14.121 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T18:36:14.121 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T18:36:14.121 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:14.121 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-17T18:36:14.128 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-17T18:36:14.151 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 1/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 3/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 4/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 5/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 6/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 7/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 8/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-17T18:36:14.536 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-17T18:36:14.537 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:14.595 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T18:36:14.596 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout: zip-3.0-35.el9.x86_64 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:14.597 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:14.632 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-base 2026-02-17T18:36:14.938 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:30 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:15.001 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:15.019 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:15.019 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:15.019 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:15.039 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install cephadm 2026-02-17T18:36:15.340 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:31 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout: cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 1.0 M 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:Install 1 Package 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 1.0 M 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 1.0 M 2026-02-17T18:36:15.421 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:16.053 INFO:teuthology.orchestra.run.trial068.stdout:cephadm-20.3.0-5230.gafefaa40.el9.noarch.rpm 1.6 MB/s | 1.0 MB 00:00 2026-02-17T18:36:16.053 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:16.053 INFO:teuthology.orchestra.run.trial068.stdout:Total 1.6 MB/s | 1.0 MB 00:00 2026-02-17T18:36:16.053 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:16.056 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:16.056 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:16.061 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:16.061 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:16.079 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:16.139 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-17T18:36:16.153 INFO:teuthology.orchestra.run.trial068.stdout: Installing : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-17T18:36:16.215 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-17T18:36:16.269 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-17T18:36:16.269 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:16.269 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:16.269 INFO:teuthology.orchestra.run.trial068.stdout: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:16.269 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:16.269 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:16.296 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T18:36:16.602 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:32 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:16.684 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:16.684 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:16.684 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repo Size 2026-02-17T18:36:16.684 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 147 k 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout:Install 1 Package 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 147 k 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 419 k 2026-02-17T18:36:16.685 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:16.769 INFO:teuthology.orchestra.run.trial068.stdout:ceph-immutable-object-cache-20.3.0-5230.gafefaa 1.7 MB/s | 147 kB 00:00 2026-02-17T18:36:16.769 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:16.769 INFO:teuthology.orchestra.run.trial068.stdout:Total 1.7 MB/s | 147 kB 00:00 2026-02-17T18:36:16.770 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:16.774 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:16.774 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:16.801 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:16.801 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:16.850 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:16.851 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-17T18:36:16.873 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-17T18:36:16.873 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:16.873 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T18:36:16.874 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:17.146 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-17T18:36:17.146 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:17.146 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:17.146 INFO:teuthology.orchestra.run.trial068.stdout: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:17.146 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:17.146 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:17.173 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-mgr 2026-02-17T18:36:17.481 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:17.544 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:17.561 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:17.562 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:17.562 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:17.582 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T18:36:17.892 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:33 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 51 M 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:17.974 INFO:teuthology.orchestra.run.trial068.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 57 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 17 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout:Install 13 Packages 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 55 M 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 262 M 2026-02-17T18:36:17.975 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:18.520 INFO:teuthology.orchestra.run.trial068.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5230.gafe 157 kB/s | 17 kB 00:00 2026-02-17T18:36:18.542 INFO:teuthology.orchestra.run.trial068.stdout:(2/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 2.2 MB/s | 48 kB 00:00 2026-02-17T18:36:18.542 INFO:teuthology.orchestra.run.trial068.stdout:(3/13): ceph-grafana-dashboards-20.3.0-5230.gaf 430 kB/s | 57 kB 00:00 2026-02-17T18:36:18.559 INFO:teuthology.orchestra.run.trial068.stdout:(4/13): python3-protobuf-3.14.0-17.el9.noarch.r 15 MB/s | 267 kB 00:00 2026-02-17T18:36:18.568 INFO:teuthology.orchestra.run.trial068.stdout:(5/13): protobuf-compiler-3.14.0-17.el9.x86_64. 33 MB/s | 862 kB 00:00 2026-02-17T18:36:18.707 INFO:teuthology.orchestra.run.trial068.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 140 kB/s | 19 kB 00:00 2026-02-17T18:36:18.746 INFO:teuthology.orchestra.run.trial068.stdout:(7/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.9 MB/s | 551 kB 00:00 2026-02-17T18:36:18.845 INFO:teuthology.orchestra.run.trial068.stdout:(8/13): python3-grpcio-1.46.7-10.el9.x86_64.rpm 15 MB/s | 2.0 MB 00:00 2026-02-17T18:36:18.949 INFO:teuthology.orchestra.run.trial068.stdout:(9/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 712 kB/s | 144 kB 00:00 2026-02-17T18:36:18.958 INFO:teuthology.orchestra.run.trial068.stdout:(10/13): python3-repoze-lru-0.7-16.el9.noarch.r 274 kB/s | 31 kB 00:00 2026-02-17T18:36:18.997 INFO:teuthology.orchestra.run.trial068.stdout:(11/13): python3-routes-2.5.1-5.el9.noarch.rpm 3.9 MB/s | 188 kB 00:00 2026-02-17T18:36:19.004 INFO:teuthology.orchestra.run.trial068.stdout:(12/13): python3-xmltodict-0.12.0-15.el9.noarch 481 kB/s | 22 kB 00:00 2026-02-17T18:36:28.560 INFO:teuthology.orchestra.run.trial068.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5230.gafefaa 5.0 MB/s | 51 MB 00:10 2026-02-17T18:36:28.562 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:28.562 INFO:teuthology.orchestra.run.trial068.stdout:Total 5.2 MB/s | 55 MB 00:10 2026-02-17T18:36:28.590 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:28.599 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:28.600 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:28.810 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:28.810 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:29.037 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:29.042 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-17T18:36:29.054 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-17T18:36:29.056 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-17T18:36:29.084 INFO:teuthology.orchestra.run.trial068.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-17T18:36:29.127 INFO:teuthology.orchestra.run.trial068.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-17T18:36:29.136 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-17T18:36:29.163 INFO:teuthology.orchestra.run.trial068.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-17T18:36:29.173 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-17T18:36:29.180 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-17T18:36:29.183 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-17T18:36:29.192 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 11/13 2026-02-17T18:36:32.069 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 12/13 2026-02-17T18:36:32.076 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 1/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 2/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 3/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-17T18:36:32.372 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-17T18:36:32.428 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-17T18:36:32.428 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:32.428 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:32.428 INFO:teuthology.orchestra.run.trial068.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:32.429 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:32.466 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T18:36:32.772 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:48 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:32.854 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout:======================================================================================= 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout:======================================================================================= 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 7.4 M 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T18:36:32.855 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout:======================================================================================= 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout:Install 12 Packages 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 43 M 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 234 M 2026-02-17T18:36:32.856 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:33.108 INFO:teuthology.orchestra.run.trial068.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 1.7 MB/s | 184 kB 00:00 2026-02-17T18:36:33.139 INFO:teuthology.orchestra.run.trial068.stdout:(2/12): libgfortran-11.5.0-14.el9.x86_64.rpm 5.7 MB/s | 794 kB 00:00 2026-02-17T18:36:33.154 INFO:teuthology.orchestra.run.trial068.stdout:(3/12): flexiblas-3.0.4-9.el9.x86_64.rpm 641 kB/s | 30 kB 00:00 2026-02-17T18:36:33.175 INFO:teuthology.orchestra.run.trial068.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 689 kB/s | 15 kB 00:00 2026-02-17T18:36:33.198 INFO:teuthology.orchestra.run.trial068.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-17T18:36:33.310 INFO:teuthology.orchestra.run.trial068.stdout:(6/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2026-02-17T18:36:33.331 INFO:teuthology.orchestra.run.trial068.stdout:(7/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 40 MB/s | 5.3 MB 00:00 2026-02-17T18:36:33.373 INFO:teuthology.orchestra.run.trial068.stdout:(8/12): python3-devel-3.9.25-2.el9.x86_64.rpm 3.8 MB/s | 244 kB 00:00 2026-02-17T18:36:33.389 INFO:teuthology.orchestra.run.trial068.stdout:(9/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 107 MB/s | 6.1 MB 00:00 2026-02-17T18:36:33.399 INFO:teuthology.orchestra.run.trial068.stdout:(10/12): python3-numpy-f2py-1.23.5-2.el9.x86_64 17 MB/s | 442 kB 00:00 2026-02-17T18:36:33.541 INFO:teuthology.orchestra.run.trial068.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 127 MB/s | 19 MB 00:00 2026-02-17T18:36:34.560 INFO:teuthology.orchestra.run.trial068.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 4.8 MB/s | 7.4 MB 00:01 2026-02-17T18:36:34.561 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:34.561 INFO:teuthology.orchestra.run.trial068.stdout:Total 25 MB/s | 43 MB 00:01 2026-02-17T18:36:34.723 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:34.736 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:34.736 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:34.937 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:34.937 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:35.157 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:35.161 INFO:teuthology.orchestra.run.trial068.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-17T18:36:35.169 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-17T18:36:35.202 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-17T18:36:35.207 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 4/12 2026-02-17T18:36:35.267 INFO:teuthology.orchestra.run.trial068.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-17T18:36:35.269 INFO:teuthology.orchestra.run.trial068.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-17T18:36:35.290 INFO:teuthology.orchestra.run.trial068.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-17T18:36:35.608 INFO:teuthology.orchestra.run.trial068.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-17T18:36:35.680 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-17T18:36:36.337 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-17T18:36:36.806 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-17T18:36:36.807 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 1/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 9/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-17T18:36:37.090 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:37.147 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:37.183 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-mgr-rook 2026-02-17T18:36:37.490 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:37.572 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-rook noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 50 k 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T18:36:37.573 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout:Install 10 Packages 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 2.0 M 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 25 M 2026-02-17T18:36:37.574 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:38.047 INFO:teuthology.orchestra.run.trial068.stdout:(1/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.4 MB/s | 159 kB 00:00 2026-02-17T18:36:38.052 INFO:teuthology.orchestra.run.trial068.stdout:(2/10): python3-pyasn1-modules-0.4.8-6.el9.noar 5.3 MB/s | 279 kB 00:00 2026-02-17T18:36:38.056 INFO:teuthology.orchestra.run.trial068.stdout:(3/10): python3-requests-oauthlib-1.3.0-12.el9. 5.4 MB/s | 54 kB 00:00 2026-02-17T18:36:38.118 INFO:teuthology.orchestra.run.trial068.stdout:(4/10): ceph-mgr-rook-20.3.0-5230.gafefaa40.el9 431 kB/s | 50 kB 00:00 2026-02-17T18:36:38.183 INFO:teuthology.orchestra.run.trial068.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 112 kB/s | 14 kB 00:00 2026-02-17T18:36:38.205 INFO:teuthology.orchestra.run.trial068.stdout:(6/10): python3-cachetools-4.2.4-1.el9.noarch.r 211 kB/s | 32 kB 00:00 2026-02-17T18:36:38.281 INFO:teuthology.orchestra.run.trial068.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 778 kB/s | 59 kB 00:00 2026-02-17T18:36:38.348 INFO:teuthology.orchestra.run.trial068.stdout:(8/10): python3-websocket-client-1.2.3-2.el9.no 1.3 MB/s | 90 kB 00:00 2026-02-17T18:36:38.352 INFO:teuthology.orchestra.run.trial068.stdout:(9/10): python3-google-auth-2.45.0-1.el9.noarch 1.1 MB/s | 254 kB 00:00 2026-02-17T18:36:38.402 INFO:teuthology.orchestra.run.trial068.stdout:(10/10): python3-kubernetes-26.1.0-3.el9.noarch 4.6 MB/s | 1.0 MB 00:00 2026-02-17T18:36:38.404 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:38.404 INFO:teuthology.orchestra.run.trial068.stdout:Total 2.4 MB/s | 2.0 MB 00:00 2026-02-17T18:36:38.429 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:38.437 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:38.437 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:38.475 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:38.476 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:38.551 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:38.602 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-17T18:36:38.614 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-17T18:36:38.629 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-17T18:36:38.634 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-17T18:36:38.641 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-17T18:36:38.681 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-17T18:36:38.693 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-17T18:36:39.012 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-17T18:36:39.026 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-17T18:36:39.027 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-17T18:36:39.134 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 1/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-17T18:36:39.135 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-17T18:36:39.190 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:39.191 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:39.220 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T18:36:39.528 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 184 k 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:39.610 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:39.611 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:39.611 INFO:teuthology.orchestra.run.trial068.stdout:Install 7 Packages 2026-02-17T18:36:39.611 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:39.611 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 7.1 M 2026-02-17T18:36:39.611 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 34 M 2026-02-17T18:36:39.611 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:39.864 INFO:teuthology.orchestra.run.trial068.stdout:(1/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 5.0 MB/s | 249 kB 00:00 2026-02-17T18:36:39.873 INFO:teuthology.orchestra.run.trial068.stdout:(2/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 3.7 MB/s | 35 kB 00:00 2026-02-17T18:36:39.913 INFO:teuthology.orchestra.run.trial068.stdout:(3/7): python3-babel-2.9.1-2.el9.noarch.rpm 61 MB/s | 6.0 MB 00:00 2026-02-17T18:36:40.100 INFO:teuthology.orchestra.run.trial068.stdout:(4/7): ceph-mgr-cephadm-20.3.0-5230.gafefaa40.e 645 kB/s | 184 kB 00:00 2026-02-17T18:36:40.213 INFO:teuthology.orchestra.run.trial068.stdout:(5/7): python3-natsort-7.1.1-5.el9.noarch.rpm 192 kB/s | 58 kB 00:00 2026-02-17T18:36:40.396 INFO:teuthology.orchestra.run.trial068.stdout:(6/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2026-02-17T18:36:40.456 INFO:teuthology.orchestra.run.trial068.stdout:(7/7): python3-typing-extensions-4.15.0-1.el9.n 243 kB/s | 86 kB 00:00 2026-02-17T18:36:40.457 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:40.458 INFO:teuthology.orchestra.run.trial068.stdout:Total 8.4 MB/s | 7.1 MB 00:00 2026-02-17T18:36:40.496 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:40.503 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:40.504 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:40.576 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:40.576 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:40.666 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:40.714 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-17T18:36:40.724 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-17T18:36:40.729 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-17T18:36:40.946 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-17T18:36:40.969 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-17T18:36:40.993 INFO:teuthology.orchestra.run.trial068.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-17T18:36:40.995 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-17T18:36:41.103 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-17T18:36:41.103 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/7 2026-02-17T18:36:41.103 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-17T18:36:41.103 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-17T18:36:41.103 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-17T18:36:41.103 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-17T18:36:41.103 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:41.160 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:41.190 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-osd 2026-02-17T18:36:41.499 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:41.563 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:41.580 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:41.581 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:41.581 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:41.602 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-osd-classic 2026-02-17T18:36:41.908 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:41.973 INFO:teuthology.orchestra.run.trial068.stdout:Package ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:41.991 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:41.991 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:41.991 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:42.012 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-fuse 2026-02-17T18:36:42.320 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repo Size 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout: ceph-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 987 k 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout:Install 1 Package 2026-02-17T18:36:42.403 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:42.404 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 987 k 2026-02-17T18:36:42.404 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 2.9 M 2026-02-17T18:36:42.404 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:43.195 INFO:teuthology.orchestra.run.trial068.stdout:ceph-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 1.2 MB/s | 987 kB 00:00 2026-02-17T18:36:43.196 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:43.196 INFO:teuthology.orchestra.run.trial068.stdout:Total 1.2 MB/s | 987 kB 00:00 2026-02-17T18:36:43.196 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:43.201 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:43.201 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:43.238 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:43.238 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:43.329 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:43.346 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:43.585 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:43.641 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:43.641 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:43.641 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:43.641 INFO:teuthology.orchestra.run.trial068.stdout: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:43.641 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:43.641 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:43.670 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install ceph-volume 2026-02-17T18:36:43.983 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repository Size 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout: ceph-volume noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 299 k 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout:Install 2 Packages 2026-02-17T18:36:44.065 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:44.066 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 649 k 2026-02-17T18:36:44.066 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 2.2 M 2026-02-17T18:36:44.066 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:44.484 INFO:teuthology.orchestra.run.trial068.stdout:(1/2): cryptsetup-2.8.1-2.el9.x86_64.rpm 3.1 MB/s | 350 kB 00:00 2026-02-17T18:36:44.661 INFO:teuthology.orchestra.run.trial068.stdout:(2/2): ceph-volume-20.3.0-5230.gafefaa40.el9.no 1.0 MB/s | 299 kB 00:00 2026-02-17T18:36:44.663 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:44.663 INFO:teuthology.orchestra.run.trial068.stdout:Total 1.1 MB/s | 649 kB 00:00 2026-02-17T18:36:44.673 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:44.679 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:44.679 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:44.723 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:44.723 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:44.804 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:44.866 INFO:teuthology.orchestra.run.trial068.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 1/2 2026-02-17T18:36:44.868 INFO:teuthology.orchestra.run.trial068.stdout: Installing : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-17T18:36:44.884 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-17T18:36:44.884 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:44.884 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T18:36:44.884 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:45.101 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 1/2 2026-02-17T18:36:45.158 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 2/2 2026-02-17T18:36:45.158 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:45.158 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:45.158 INFO:teuthology.orchestra.run.trial068.stdout: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T18:36:45.158 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:45.158 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:45.187 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install librados-devel 2026-02-17T18:36:45.500 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repo Size 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout: librados-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 123 k 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:45.582 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:45.583 INFO:teuthology.orchestra.run.trial068.stdout:Install 1 Package 2026-02-17T18:36:45.583 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:45.583 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 123 k 2026-02-17T18:36:45.583 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 437 k 2026-02-17T18:36:45.583 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:45.874 INFO:teuthology.orchestra.run.trial068.stdout:librados-devel-20.3.0-5230.gafefaa40.el9.x86_64 423 kB/s | 123 kB 00:00 2026-02-17T18:36:45.874 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:45.875 INFO:teuthology.orchestra.run.trial068.stdout:Total 421 kB/s | 123 kB 00:00 2026-02-17T18:36:45.875 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:45.878 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:45.878 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:45.909 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:45.909 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:45.964 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:45.980 INFO:teuthology.orchestra.run.trial068.stdout: Installing : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:46.067 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:46.124 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:46.124 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:46.124 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:46.124 INFO:teuthology.orchestra.run.trial068.stdout: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:46.124 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:46.125 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:46.152 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install libcephfs2 2026-02-17T18:36:46.458 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:46.522 INFO:teuthology.orchestra.run.trial068.stdout:Package libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:46.540 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:46.541 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:46.541 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:46.561 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install libcephfs-devel 2026-02-17T18:36:46.868 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:46.951 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repo Size 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 37 k 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout:Installing dependencies: 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:46.952 INFO:teuthology.orchestra.run.trial068.stdout:Install 2 Packages 2026-02-17T18:36:46.953 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:46.953 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 62 k 2026-02-17T18:36:46.953 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 223 k 2026-02-17T18:36:46.953 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:47.250 INFO:teuthology.orchestra.run.trial068.stdout:(1/2): libcephfs-proxy2-20.3.0-5230.gafefaa40.e 83 kB/s | 25 kB 00:00 2026-02-17T18:36:47.253 INFO:teuthology.orchestra.run.trial068.stdout:(2/2): libcephfs-devel-20.3.0-5230.gafefaa40.el 124 kB/s | 37 kB 00:00 2026-02-17T18:36:47.254 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:47.254 INFO:teuthology.orchestra.run.trial068.stdout:Total 205 kB/s | 62 kB 00:00 2026-02-17T18:36:47.254 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:47.257 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:47.257 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:47.290 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:47.290 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:47.339 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:47.340 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-17T18:36:47.354 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-17T18:36:47.369 INFO:teuthology.orchestra.run.trial068.stdout: Installing : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-17T18:36:47.421 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-17T18:36:47.421 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-17T18:36:47.477 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-17T18:36:47.478 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:47.478 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:47.478 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:47.478 INFO:teuthology.orchestra.run.trial068.stdout: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:47.478 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:47.478 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:47.503 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install librados2 2026-02-17T18:36:47.810 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:03 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:47.874 INFO:teuthology.orchestra.run.trial068.stdout:Package librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:47.891 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:47.892 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:47.892 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:47.913 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install librbd1 2026-02-17T18:36:48.222 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:48.287 INFO:teuthology.orchestra.run.trial068.stdout:Package librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:48.304 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:48.304 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:48.305 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:48.326 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install python3-rados 2026-02-17T18:36:48.636 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:48.701 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:48.719 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:48.719 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:48.719 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:48.740 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install python3-rgw 2026-02-17T18:36:49.050 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:49.115 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:49.133 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:49.133 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:49.133 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:49.155 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install python3-cephfs 2026-02-17T18:36:49.464 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:49.529 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:49.546 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:49.547 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:49.547 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:49.568 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install python3-rbd 2026-02-17T18:36:49.880 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:49.945 INFO:teuthology.orchestra.run.trial068.stdout:Package python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-17T18:36:49.963 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:49.963 INFO:teuthology.orchestra.run.trial068.stdout:Nothing to do. 2026-02-17T18:36:49.963 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:49.984 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install rbd-fuse 2026-02-17T18:36:50.293 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:06 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:50.376 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:50.376 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:50.376 INFO:teuthology.orchestra.run.trial068.stdout: Package Architecture Version Repository Size 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout: rbd-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 84 k 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:Install 1 Package 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 84 k 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 209 k 2026-02-17T18:36:50.377 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:50.608 INFO:teuthology.orchestra.run.trial068.stdout:rbd-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 365 kB/s | 84 kB 00:00 2026-02-17T18:36:50.609 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:50.609 INFO:teuthology.orchestra.run.trial068.stdout:Total 363 kB/s | 84 kB 00:00 2026-02-17T18:36:50.609 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:50.613 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:50.613 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:50.644 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:50.644 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:50.694 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:50.711 INFO:teuthology.orchestra.run.trial068.stdout: Installing : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:50.787 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:50.845 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:50.845 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:50.845 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:50.845 INFO:teuthology.orchestra.run.trial068.stdout: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:50.845 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:50.845 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:50.873 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install rbd-mirror 2026-02-17T18:36:51.184 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:07 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout: Package Arch Version Repo Size 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout: rbd-mirror x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.9 M 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:Install 1 Package 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 2.9 M 2026-02-17T18:36:51.267 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 11 M 2026-02-17T18:36:51.268 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:51.662 INFO:teuthology.orchestra.run.trial068.stdout:rbd-mirror-20.3.0-5230.gafefaa40.el9.x86_64.rpm 7.3 MB/s | 2.9 MB 00:00 2026-02-17T18:36:51.663 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:51.664 INFO:teuthology.orchestra.run.trial068.stdout:Total 7.2 MB/s | 2.9 MB 00:00 2026-02-17T18:36:51.664 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:51.668 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:51.668 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:51.707 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:51.707 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:51.883 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:51.884 INFO:teuthology.orchestra.run.trial068.stdout: Installing : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:51.908 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:51.908 INFO:teuthology.orchestra.run.trial068.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T18:36:51.908 INFO:teuthology.orchestra.run.trial068.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T18:36:51.908 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T18:36:51.908 INFO:teuthology.orchestra.run.trial068.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T18:36:51.908 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:52.199 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:52.199 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:52.199 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:52.200 INFO:teuthology.orchestra.run.trial068.stdout: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:52.200 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:52.200 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:52.227 DEBUG:teuthology.orchestra.run.trial068:> sudo yum -y install rbd-nbd 2026-02-17T18:36:52.542 INFO:teuthology.orchestra.run.trial068.stdout:Last metadata expiration check: 0:01:08 ago on Tue 17 Feb 2026 06:35:44 PM UTC. 2026-02-17T18:36:52.624 INFO:teuthology.orchestra.run.trial068.stdout:Dependencies resolved. 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout: Package Architecture Version Repository Size 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:Installing: 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout: rbd-nbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 172 k 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:Transaction Summary 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:================================================================================ 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:Install 1 Package 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:Total download size: 172 k 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:Installed size: 462 k 2026-02-17T18:36:52.625 INFO:teuthology.orchestra.run.trial068.stdout:Downloading Packages: 2026-02-17T18:36:52.721 INFO:teuthology.orchestra.run.trial068.stdout:rbd-nbd-20.3.0-5230.gafefaa40.el9.x86_64.rpm 1.8 MB/s | 172 kB 00:00 2026-02-17T18:36:52.721 INFO:teuthology.orchestra.run.trial068.stdout:-------------------------------------------------------------------------------- 2026-02-17T18:36:52.722 INFO:teuthology.orchestra.run.trial068.stdout:Total 1.7 MB/s | 172 kB 00:00 2026-02-17T18:36:52.722 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction check 2026-02-17T18:36:52.726 INFO:teuthology.orchestra.run.trial068.stdout:Transaction check succeeded. 2026-02-17T18:36:52.727 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction test 2026-02-17T18:36:52.757 INFO:teuthology.orchestra.run.trial068.stdout:Transaction test succeeded. 2026-02-17T18:36:52.758 INFO:teuthology.orchestra.run.trial068.stdout:Running transaction 2026-02-17T18:36:52.813 INFO:teuthology.orchestra.run.trial068.stdout: Preparing : 1/1 2026-02-17T18:36:52.829 INFO:teuthology.orchestra.run.trial068.stdout: Installing : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:52.912 INFO:teuthology.orchestra.run.trial068.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:52.969 INFO:teuthology.orchestra.run.trial068.stdout: Verifying : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-17T18:36:52.970 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:52.970 INFO:teuthology.orchestra.run.trial068.stdout:Installed: 2026-02-17T18:36:52.970 INFO:teuthology.orchestra.run.trial068.stdout: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-17T18:36:52.970 INFO:teuthology.orchestra.run.trial068.stdout: 2026-02-17T18:36:52.970 INFO:teuthology.orchestra.run.trial068.stdout:Complete! 2026-02-17T18:36:53.000 DEBUG:teuthology.parallel:result is None 2026-02-17T18:36:53.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-17T18:36:53.122 DEBUG:teuthology.orchestra.run.trial068:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T18:36:53.144 INFO:teuthology.orchestra.run.trial068.stdout:20.3.0-5230.gafefaa40.el9 2026-02-17T18:36:53.144 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5230.gafefaa40.el9 2026-02-17T18:36:53.144 INFO:teuthology.task.install:The correct ceph version 20.3.0-5230.gafefaa40 is installed. 2026-02-17T18:36:53.146 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T18:36:53.146 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-17T18:36:53.146 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T18:36:53.209 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T18:36:53.209 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-17T18:36:53.209 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T18:36:53.275 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T18:36:53.339 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T18:36:53.339 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-17T18:36:53.339 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T18:36:53.401 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T18:36:53.464 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T18:36:53.464 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-17T18:36:53.464 DEBUG:teuthology.orchestra.run.trial068:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T18:36:53.526 DEBUG:teuthology.orchestra.run.trial068:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T18:36:53.589 INFO:teuthology.run_tasks:Running task workunit... 2026-02-17T18:36:53.604 INFO:tasks.workunit:Pulling workunits from ref afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-17T18:36:53.604 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-17T18:36:53.604 INFO:tasks.workunit:timeout=3h 2026-02-17T18:36:53.604 INFO:tasks.workunit:cleanup=True 2026-02-17T18:36:53.604 DEBUG:teuthology.orchestra.run.trial068:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-17T18:36:53.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T18:36:53.646 INFO:teuthology.orchestra.run.trial068.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-17T18:36:53.646 DEBUG:teuthology.orchestra.run.trial068:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T18:36:53.703 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-17T18:36:53.703 DEBUG:teuthology.orchestra.run.trial068:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-17T18:36:53.760 DEBUG:teuthology.orchestra.run.trial068:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-17T18:36:53.823 INFO:tasks.workunit.client.0.trial068.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-17T18:37:24.825 INFO:tasks.workunit.client.0.trial068.stderr:Note: switching to 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'. 2026-02-17T18:37:24.825 INFO:tasks.workunit.client.0.trial068.stderr: 2026-02-17T18:37:24.825 INFO:tasks.workunit.client.0.trial068.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr:state without impacting any branches by switching back to a branch. 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr: 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr: 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr: git switch -c 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr: 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr:Or undo this operation with: 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr: 2026-02-17T18:37:24.826 INFO:tasks.workunit.client.0.trial068.stderr: git switch - 2026-02-17T18:37:24.827 INFO:tasks.workunit.client.0.trial068.stderr: 2026-02-17T18:37:24.827 INFO:tasks.workunit.client.0.trial068.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-17T18:37:24.827 INFO:tasks.workunit.client.0.trial068.stderr: 2026-02-17T18:37:24.827 INFO:tasks.workunit.client.0.trial068.stderr:HEAD is now at afefaa40503 Merge branch 'block_non_4k_chunks' of https://github.com/aainscow/ceph into wip-yuri4-testing-2026-02-11-2126 2026-02-17T18:37:24.832 DEBUG:teuthology.orchestra.run.trial068:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-17T18:37:24.892 DEBUG:teuthology.orchestra.run.trial068:> set -ex 2026-02-17T18:37:24.892 DEBUG:teuthology.orchestra.run.trial068:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-17T18:37:24.948 INFO:tasks.workunit:Running workunits matching misc on client.0... 2026-02-17T18:37:24.948 INFO:tasks.workunit:Running workunit misc/mclock-config.sh... 2026-02-17T18:37:24.948 DEBUG:teuthology.orchestra.run.trial068:workunit test misc/mclock-config.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh 2026-02-17T18:37:25.010 INFO:tasks.workunit.client.0.trial068.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/mclock-config 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:21: run: local dir=td/mclock-config 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:22: run: shift 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: export CEPH_MON=127.0.0.1:7124 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: CEPH_MON=127.0.0.1:7124 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:25: run: export CEPH_ARGS 2026-02-17T18:37:25.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: uuidgen 2026-02-17T18:37:25.015 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: CEPH_ARGS+='--fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none ' 2026-02-17T18:37:25.015 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-17T18:37:25.015 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:28: run: CEPH_ARGS+='--debug-mclock 20 ' 2026-02-17T18:37:25.016 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: set 2026-02-17T18:37:25.016 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: local 'funcs=TEST_backfill_limit_adjustment_mclock 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:TEST_profile_builtin_to_custom 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:TEST_profile_custom_to_builtin 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:TEST_recovery_limit_adjustment_mclock' 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T18:37:25.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:25.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:25.019 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:25.020 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:25.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:25.021 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:25.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:25.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:25.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:25.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:25.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:25.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:25.023 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:25.024 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:25.024 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:25.024 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:25.026 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:25.026 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.026 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.026 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:25.026 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:25.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:25.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T18:37:25.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:25.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.028 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.18341 2026-02-17T18:37:25.029 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_backfill_limit_adjustment_mclock td/mclock-config 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: setup td/mclock-config 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:25.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:25.032 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:25.032 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:25.032 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:25.033 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:25.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:25.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:25.034 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:25.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:25.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:25.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:25.035 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:25.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:25.037 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:25.037 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:25.038 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:25.038 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.038 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:25.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:25.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:25.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T18:37:25.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:25.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.18341 2026-02-17T18:37:25.041 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:239: TEST_backfill_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T18:37:25.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T18:37:25.068 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:37:25.069 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:25.069 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:25.069 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:25.069 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.069 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.069 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:25.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:37:25.091 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:37:25.092 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:37:25.092 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:25.092 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:25.092 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.093 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:25.094 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:25.094 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get fsid 2026-02-17T18:37:25.136 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.137 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:25.138 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:25.138 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get mon_host 2026-02-17T18:37:25.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:240: TEST_backfill_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-17T18:37:25.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T18:37:25.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:37:25.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:37:25.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:37:25.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T18:37:25.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:37:25.280 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:37:25.280 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:25.280 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:25.280 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:25.280 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.280 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.280 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:25.281 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:25.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:242: TEST_backfill_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T18:37:25.297 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T18:37:25.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T18:37:25.299 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:37:25.299 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=efcb0435-f516-44fe-afcb-d37e872ecb99 2026-02-17T18:37:25.299 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 efcb0435-f516-44fe-afcb-d37e872ecb99' 2026-02-17T18:37:25.299 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 efcb0435-f516-44fe-afcb-d37e872ecb99 2026-02-17T18:37:25.299 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:37:25.307 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBltZRpDVk9EhAA2CQfAsfhwatyJ9jCNQlVNA== 2026-02-17T18:37:25.307 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBltZRpDVk9EhAA2CQfAsfhwatyJ9jCNQlVNA=="}' 2026-02-17T18:37:25.307 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new efcb0435-f516-44fe-afcb-d37e872ecb99 -i td/mclock-config/0/new.json 2026-02-17T18:37:25.410 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:25.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T18:37:25.417 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQBltZRpDVk9EhAA2CQfAsfhwatyJ9jCNQlVNA== --osd-uuid efcb0435-f516-44fe-afcb-d37e872ecb99 2026-02-17T18:37:25.433 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.431+0000 7fd3c320f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:25.434 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.433+0000 7fd3c320f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:25.435 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.434+0000 7fd3c320f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:25.435 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.434+0000 7fd3c320f780 -1 bdev(0x3c38c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:37:25.435 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.434+0000 7fd3c320f780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T18:37:25.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T18:37:25.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:37:25.866 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:37:25.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:37:25.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:37:25.967 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:37:25.967 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:37:25.968 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T18:37:25.968 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:37:25.968 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:37:25.968 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:37:25.981 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.979+0000 7fad0e1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:25.982 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.981+0000 7fad0e1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:25.983 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:25.982+0000 7fad0e1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:26.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:26.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:37:26.168 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:26.167+0000 7fad0e1d8780 -1 Falling back to public interface 2026-02-17T18:37:26.520 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:26.519+0000 7fad0e1d8780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:37:27.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:37:27.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:27.156 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:27.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:37:27.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:27.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:27.249 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3131917698,v1:127.0.0.1:6801/3131917698] [v2:127.0.0.1:6802/3131917698,v1:127.0.0.1:6803/3131917698] exists,up efcb0435-f516-44fe-afcb-d37e872ecb99 2026-02-17T18:37:27.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:27.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:27.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:27.250 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T18:37:27.250 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:27.250 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:27.251 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:37:27.299 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: local backfills=1 2026-02-17T18:37:27.300 INFO:tasks.workunit.client.0.trial068.stdout:osd_max_backfills: 1 2026-02-17T18:37:27.300 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: echo 'osd_max_backfills: 1' 2026-02-17T18:37:27.300 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:251: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-17T18:37:27.400 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:252: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-17T18:37:29.402 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T18:37:29.402 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:29.403 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:29.403 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:29.403 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:29.403 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:29.403 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:29.403 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:29.404 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:29.404 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:29.404 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:37:29.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: local max_backfills=1 2026-02-17T18:37:29.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-17T18:37:29.451 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-17T18:37:29.451 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:29.453 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T18:37:29.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-17T18:37:29.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:260: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:29.499 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:29.500 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T18:37:29.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-17T18:37:29.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:263: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-17T18:37:29.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:267: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-17T18:37:29.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:268: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-17T18:37:29.988 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:269: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-17T18:37:31.990 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T18:37:31.990 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:31.991 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:31.991 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:31.991 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:31.992 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:31.992 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:31.992 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:31.992 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:31.992 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:31.992 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:37:32.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T18:37:32.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:272: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T18:37:32.039 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:32.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T18:37:32.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T18:37:32.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:277: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T18:37:32.085 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-17T18:37:32.085 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:32.086 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T18:37:32.132 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T18:37:32.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:280: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T18:37:32.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:284: TEST_backfill_limit_adjustment_mclock: kill_daemons td/mclock-config TERM osd 2026-02-17T18:37:32.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:32.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:32.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:32.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:32.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:32.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:32.238 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:285: TEST_backfill_limit_adjustment_mclock: ceph osd down 0 2026-02-17T18:37:32.471 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 is already down. 2026-02-17T18:37:32.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:286: TEST_backfill_limit_adjustment_mclock: wait_for_osd down 0 2026-02-17T18:37:32.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-17T18:37:32.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:37:32.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:32.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:32.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:32.480 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:32.481 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:32.481 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:32.481 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-17T18:37:32.702 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3131917698,v1:127.0.0.1:6801/3131917698] [v2:127.0.0.1:6802/3131917698,v1:127.0.0.1:6803/3131917698] exists efcb0435-f516-44fe-afcb-d37e872ecb99 2026-02-17T18:37:32.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:32.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:32.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:32.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:287: TEST_backfill_limit_adjustment_mclock: activate_osd td/mclock-config 0 --osd-op-queue=mclock_scheduler 2026-02-17T18:37:32.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/mclock-config 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/mclock-config/0 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T18:37:32.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:32.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=--osd-op-queue=mclock_scheduler 2026-02-17T18:37:32.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/mclock-config/0 2026-02-17T18:37:32.710 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:37:32.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T18:37:32.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler 2026-02-17T18:37:32.711 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/mclock-config/0/whoami 2026-02-17T18:37:32.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T18:37:32.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T18:37:32.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T18:37:32.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T18:37:32.718 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:32.716+0000 7f0bbc83b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:32.720 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:32.717+0000 7f0bbc83b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:32.721 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:32.718+0000 7f0bbc83b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:32.852 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:32.849+0000 7f0bbc83b780 -1 Falling back to public interface 2026-02-17T18:37:32.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T18:37:32.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:32.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:37:32.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:32.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:32.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:32.937 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:32.937 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:32.937 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:32.937 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:33.157 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:33.154+0000 7f0bbc83b780 -1 osd.0 6 log_to_monitors true 2026-02-17T18:37:33.164 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:37:34.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:37:34.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:34.166 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:34.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:37:34.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:34.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:34.390 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 6 last_clean_interval [5,6) [v2:127.0.0.1:6800/397895906,v1:127.0.0.1:6801/397895906] [v2:127.0.0.1:6802/397895906,v1:127.0.0.1:6803/397895906] exists,up efcb0435-f516-44fe-afcb-d37e872ecb99 2026-02-17T18:37:34.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:34.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:34.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:34.392 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:34.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:37:34.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T18:37:34.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:291: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T18:37:34.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.440 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:34.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:34.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T18:37:34.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T18:37:34.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:296: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T18:37:34.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-17T18:37:34.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: bc 2026-02-17T18:37:34.488 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:34.488 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:34.488 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:34.488 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:34.489 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.489 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.489 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:34.489 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:34.489 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok dump_recovery_reservations 2026-02-17T18:37:34.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-17T18:37:34.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:299: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-17T18:37:34.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:301: TEST_backfill_limit_adjustment_mclock: teardown td/mclock-config 2026-02-17T18:37:34.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:34.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:34.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:34.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:34.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:34.538 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:34.538 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:34.538 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:34.643 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:34.644 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:34.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:34.645 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:34.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:34.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:34.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:34.647 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:34.647 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:34.648 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:34.649 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:34.649 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:34.649 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:34.649 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:34.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:34.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:34.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:34.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:34.656 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:34.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:34.657 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:34.658 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:34.658 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:34.659 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:34.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:34.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:34.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:34.660 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:34.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:34.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:34.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:34.663 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:34.663 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.663 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:34.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:34.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:34.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:34.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:34.666 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:34.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:34.667 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:34.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:34.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:34.668 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:34.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:34.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:34.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:34.670 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:34.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:34.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:34.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:34.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:34.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:34.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:34.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:34.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T18:37:34.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:34.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.675 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.18341 2026-02-17T18:37:34.675 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:34.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:37:34.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:34.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T18:37:34.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_builtin_to_custom td/mclock-config 2026-02-17T18:37:34.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:39: TEST_profile_builtin_to_custom: local dir=td/mclock-config 2026-02-17T18:37:34.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:41: TEST_profile_builtin_to_custom: run_mon td/mclock-config a 2026-02-17T18:37:34.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T18:37:34.678 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:37:34.678 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:37:34.678 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:37:34.678 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T18:37:34.678 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T18:37:34.693 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:37:34.693 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:34.693 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:34.694 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:34.694 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.694 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:34.694 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:37:34.717 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:37:34.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:37:34.719 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:34.719 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:34.719 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:37:34.719 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:37:34.719 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:34.719 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:34.719 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:34.720 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:34.720 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.720 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.720 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:34.720 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:34.721 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get fsid 2026-02-17T18:37:34.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:37:34.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:34.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:34.762 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:37:34.762 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:37:34.762 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:34.762 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:34.762 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:34.763 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:34.763 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.763 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.763 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:34.763 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:34.763 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get mon_host 2026-02-17T18:37:34.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:42: TEST_profile_builtin_to_custom: run_mgr td/mclock-config x 2026-02-17T18:37:34.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T18:37:34.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:37:34.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:37:34.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:37:34.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T18:37:34.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:37:34.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:37:34.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:34.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:34.905 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:34.905 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.905 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.905 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:34.905 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:34.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:43: TEST_profile_builtin_to_custom: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T18:37:34.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:34.924 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:34.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:34.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:37:34.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T18:37:34.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T18:37:34.926 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:37:34.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c4c3b676-c80f-4ec5-a572-684e46fb4fe4 2026-02-17T18:37:34.926 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 c4c3b676-c80f-4ec5-a572-684e46fb4fe4 2026-02-17T18:37:34.927 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c4c3b676-c80f-4ec5-a572-684e46fb4fe4' 2026-02-17T18:37:34.927 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:37:34.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQButZRp9ZB4NxAAdcnyK3T2KBZ1ejcLAwmyHw== 2026-02-17T18:37:34.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQButZRp9ZB4NxAAdcnyK3T2KBZ1ejcLAwmyHw=="}' 2026-02-17T18:37:34.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c4c3b676-c80f-4ec5-a572-684e46fb4fe4 -i td/mclock-config/0/new.json 2026-02-17T18:37:35.032 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:35.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T18:37:35.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQButZRp9ZB4NxAAdcnyK3T2KBZ1ejcLAwmyHw== --osd-uuid c4c3b676-c80f-4ec5-a572-684e46fb4fe4 2026-02-17T18:37:35.055 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.052+0000 7f9884450780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:35.056 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.053+0000 7f9884450780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:35.057 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.054+0000 7f9884450780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:35.057 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.054+0000 7f9884450780 -1 bdev(0x2cdac00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:37:35.058 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.054+0000 7f9884450780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T18:37:35.470 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T18:37:35.471 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:37:35.471 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:37:35.472 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:37:35.472 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:37:35.574 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:37:35.574 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:37:35.575 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T18:37:35.575 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:37:35.575 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:37:35.575 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:37:35.587 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.584+0000 7fae80e13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:35.589 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.586+0000 7fae80e13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:35.589 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.587+0000 7fae80e13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:35.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:37:35.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:35.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:37:35.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:35.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:35.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:35.670 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:35.764 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:37:35.856 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:35.854+0000 7fae80e13780 -1 Falling back to public interface 2026-02-17T18:37:36.253 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:36.250+0000 7fae80e13780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:37:36.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:37:36.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:36.765 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:36.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:37:36.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:36.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:36.859 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3560125155,v1:127.0.0.1:6801/3560125155] [v2:127.0.0.1:6802/3560125155,v1:127.0.0.1:6803/3560125155] exists,up c4c3b676-c80f-4ec5-a572-684e46fb4fe4 2026-02-17T18:37:36.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:36.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:36.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:36.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2026-02-17T18:37:36.958 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: local mclock_profile=balanced 2026-02-17T18:37:36.958 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: test balanced = balanced 2026-02-17T18:37:36.958 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T18:37:36.959 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T18:37:37.006 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: mclock_profile='"high_recovery_ops"' 2026-02-17T18:37:37.007 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: eval echo '"high_recovery_ops"' 2026-02-17T18:37:37.007 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: echo high_recovery_ops 2026-02-17T18:37:37.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: mclock_profile=high_recovery_ops 2026-02-17T18:37:37.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_recovery_ops = high_recovery_ops 2026-02-17T18:37:37.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:57: TEST_profile_builtin_to_custom: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-17T18:37:37.065 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:37:37.065 INFO:tasks.workunit.client.0.trial068.stdout: "success": "" 2026-02-17T18:37:37.065 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:37:37.071 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-17T18:37:37.072 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T18:37:37.072 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:37.072 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:37.072 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:37.072 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.072 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.072 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:37.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T18:37:37.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T18:37:37.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: mclock_profile='"custom"' 2026-02-17T18:37:37.117 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: eval echo '"custom"' 2026-02-17T18:37:37.117 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: echo custom 2026-02-17T18:37:37.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: mclock_profile=custom 2026-02-17T18:37:37.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: test custom = custom 2026-02-17T18:37:37.117 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: bc 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T18:37:37.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T18:37:37.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: local client_res=0.000000 2026-02-17T18:37:37.168 INFO:tasks.workunit.client.0.trial068.stdout:client_res = 0.000000 2026-02-17T18:37:37.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:69: TEST_profile_builtin_to_custom: echo 'client_res = 0.000000' 2026-02-17T18:37:37.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: echo '0.000000 + 0.1' 2026-02-17T18:37:37.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: bc -l 2026-02-17T18:37:37.169 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: local client_res_new=.100000 2026-02-17T18:37:37.169 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:71: TEST_profile_builtin_to_custom: echo 'client_res_new = .100000' 2026-02-17T18:37:37.170 INFO:tasks.workunit.client.0.trial068.stdout:client_res_new = .100000 2026-02-17T18:37:37.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: ceph config set osd.0 osd_mclock_scheduler_client_res .100000 2026-02-17T18:37:37.397 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T18:37:37.622 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: local res=0.100000 2026-02-17T18:37:37.622 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-17T18:37:37.623 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: bc -l 2026-02-17T18:37:37.624 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-17T18:37:37.624 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: bc 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-17T18:37:37.625 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T18:37:37.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: res=0.100000 2026-02-17T18:37:37.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-17T18:37:37.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc -l 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:89: TEST_profile_builtin_to_custom: teardown td/mclock-config 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:37.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:37.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:37.780 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:37.781 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:37.781 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:37.782 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:37.783 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:37.783 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:37.784 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:37.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:37.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:37.785 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:37.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:37.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:37.790 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:37.790 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.790 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:37.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:37.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:37.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:37.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:37.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:37.794 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:37.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:37.795 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:37.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:37.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:37.796 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:37.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:37.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:37.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:37.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:37.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:37.799 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T18:37:37.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:37.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:37.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:37.803 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:37.804 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:37.804 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:37.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:37.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:37.805 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:37.806 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.806 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:37.806 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:37.806 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.807 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:37.808 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:37.808 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:37.809 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:37.809 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.809 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:37.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:37.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:37.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T18:37:37.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:37.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.18341 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_custom_to_builtin td/mclock-config 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:93: TEST_profile_custom_to_builtin: local dir=td/mclock-config 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:95: TEST_profile_custom_to_builtin: setup td/mclock-config 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:37.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:37.814 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:37.814 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:37.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:37.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:37.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:37.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:37.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:37.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:37.816 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:37.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:37.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:37.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:37.818 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.818 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:37.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:37.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:37.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:37.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:37.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:37.821 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:37.821 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.821 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:37.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:37.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:37.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T18:37:37.823 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:37.823 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.823 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.18341 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:96: TEST_profile_custom_to_builtin: run_mon td/mclock-config a 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:37:37.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:37:37.826 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T18:37:37.826 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T18:37:37.842 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:37:37.842 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:37.842 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:37.843 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:37.843 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.843 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.843 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:37.843 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:37:37.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:37:37.867 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:37:37.867 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:37.867 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:37.867 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:37:37.867 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:37.868 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:37.869 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get fsid 2026-02-17T18:37:37.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:37:37.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:37.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:37.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:37:37.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:37:37.913 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:37.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get mon_host 2026-02-17T18:37:37.955 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:97: TEST_profile_custom_to_builtin: run_mgr td/mclock-config x 2026-02-17T18:37:37.955 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T18:37:37.955 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:37:37.955 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:37:37.955 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:37:37.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T18:37:37.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:37:38.055 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:37:38.055 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:38.055 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:38.056 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:38.056 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:38.056 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:38.056 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:38.056 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:38.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:38.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:98: TEST_profile_custom_to_builtin: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:38.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:38.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:37:38.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7a07ea0d-b6ad-4791-b9cc-f86c4f2a7d24 2026-02-17T18:37:38.076 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 7a07ea0d-b6ad-4791-b9cc-f86c4f2a7d24 2026-02-17T18:37:38.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7a07ea0d-b6ad-4791-b9cc-f86c4f2a7d24' 2026-02-17T18:37:38.076 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:37:38.082 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBytZRpi4zFBBAAsxPYDj5GGEVnYKy2n5Dcig== 2026-02-17T18:37:38.082 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBytZRpi4zFBBAAsxPYDj5GGEVnYKy2n5Dcig=="}' 2026-02-17T18:37:38.082 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7a07ea0d-b6ad-4791-b9cc-f86c4f2a7d24 -i td/mclock-config/0/new.json 2026-02-17T18:37:38.183 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:38.189 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T18:37:38.190 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQBytZRpi4zFBBAAsxPYDj5GGEVnYKy2n5Dcig== --osd-uuid 7a07ea0d-b6ad-4791-b9cc-f86c4f2a7d24 2026-02-17T18:37:38.205 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.202+0000 7fe85dc0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:38.206 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.203+0000 7fe85dc0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:38.207 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.204+0000 7fe85dc0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:38.207 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.204+0000 7fe85dc0f780 -1 bdev(0x3750c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:37:38.208 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.204+0000 7fe85dc0f780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T18:37:38.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T18:37:38.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:37:38.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:37:38.607 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:37:38.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:37:38.709 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:37:38.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:37:38.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T18:37:38.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:37:38.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:37:38.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:37:38.723 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.721+0000 7f02cb837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:38.725 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.722+0000 7f02cb837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:38.726 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.723+0000 7f02cb837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:38.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:37:38.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:38.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:37:38.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:38.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:38.806 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:38.806 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:38.806 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:38.807 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:38.807 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:38.899 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:37:38.910 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:38.908+0000 7f02cb837780 -1 Falling back to public interface 2026-02-17T18:37:39.255 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:39.252+0000 7f02cb837780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:37:39.900 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:37:39.900 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:39.901 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:39.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:37:39.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:39.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:39.995 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3683031160,v1:127.0.0.1:6801/3683031160] [v2:127.0.0.1:6802/3683031160,v1:127.0.0.1:6803/3683031160] exists,up 7a07ea0d-b6ad-4791-b9cc-f86c4f2a7d24 2026-02-17T18:37:39.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:39.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:39.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:39.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: local def_mclock_profile 2026-02-17T18:37:39.997 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2026-02-17T18:37:40.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: def_mclock_profile=balanced 2026-02-17T18:37:40.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:103: TEST_profile_custom_to_builtin: test balanced = balanced 2026-02-17T18:37:40.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-17T18:37:40.097 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T18:37:40.097 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:40.098 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:40.098 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:40.098 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:40.098 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:40.098 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:40.098 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T18:37:40.098 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T18:37:40.144 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: local 'orig_mclock_profile="high_recovery_ops"' 2026-02-17T18:37:40.146 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: eval echo '"high_recovery_ops"' 2026-02-17T18:37:40.146 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: echo high_recovery_ops 2026-02-17T18:37:40.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: orig_mclock_profile=high_recovery_ops 2026-02-17T18:37:40.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:110: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-17T18:37:40.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:113: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-17T18:37:40.201 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:37:40.202 INFO:tasks.workunit.client.0.trial068.stdout: "success": "" 2026-02-17T18:37:40.202 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:40.208 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:40.209 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T18:37:40.209 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T18:37:40.254 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: local 'mclock_profile="custom"' 2026-02-17T18:37:40.254 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: eval echo '"custom"' 2026-02-17T18:37:40.254 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: echo custom 2026-02-17T18:37:40.255 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: mclock_profile=custom 2026-02-17T18:37:40.255 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:120: TEST_profile_custom_to_builtin: test custom = custom 2026-02-17T18:37:40.255 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: local client_res 2026-02-17T18:37:40.256 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T18:37:40.256 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: bc 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T18:37:40.257 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T18:37:40.301 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: client_res=0.000000 2026-02-17T18:37:40.301 INFO:tasks.workunit.client.0.trial068.stdout:Original client_res for osd.0 = 0.000000 2026-02-17T18:37:40.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.0 = 0.000000' 2026-02-17T18:37:40.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: echo '0.000000 + 0.1' 2026-02-17T18:37:40.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: bc -l 2026-02-17T18:37:40.303 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: local client_res_new=.100000 2026-02-17T18:37:40.303 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:131: TEST_profile_custom_to_builtin: echo 'client_res_new = .100000' 2026-02-17T18:37:40.303 INFO:tasks.workunit.client.0.trial068.stdout:client_res_new = .100000 2026-02-17T18:37:40.303 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_scheduler_client_res .100000 2026-02-17T18:37:40.528 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T18:37:40.748 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-17T18:37:40.749 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T18:37:40.749 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: bc -l 2026-02-17T18:37:40.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T18:37:40.751 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: bc 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T18:37:40.752 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T18:37:40.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: res=0.100000 2026-02-17T18:37:40.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T18:37:40.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: bc -l 2026-02-17T18:37:40.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T18:37:40.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:154: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile high_recovery_ops 2026-02-17T18:37:40.855 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:37:40.855 INFO:tasks.workunit.client.0.trial068.stdout: "success": "" 2026-02-17T18:37:40.855 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:37:40.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-17T18:37:40.862 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T18:37:40.862 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:40.862 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:40.862 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:40.863 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:40.863 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:40.863 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:40.863 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T18:37:40.863 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_profile 2026-02-17T18:37:40.908 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: eval 'mclock_profile="high_recovery_ops"' 2026-02-17T18:37:40.908 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: mclock_profile=high_recovery_ops 2026-02-17T18:37:40.909 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-17T18:37:40.909 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T18:37:41.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-17T18:37:41.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T18:37:41.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: bc -l 2026-02-17T18:37:41.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T18:37:41.134 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: bc 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T18:37:41.135 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T18:37:41.180 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: res=0.100000 2026-02-17T18:37:41.181 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-17T18:37:41.181 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc -l 2026-02-17T18:37:41.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T18:37:41.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:178: TEST_profile_custom_to_builtin: ceph config rm osd osd_mclock_scheduler_client_res 2026-02-17T18:37:41.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:180: TEST_profile_custom_to_builtin: sleep 5 2026-02-17T18:37:46.404 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-17T18:37:46.628 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: res=0.000000 2026-02-17T18:37:46.629 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: echo '0.000000 != 0.0' 2026-02-17T18:37:46.629 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc -l 2026-02-17T18:37:46.629 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T18:37:46.631 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-17T18:37:46.631 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: bc 2026-02-17T18:37:46.631 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-17T18:37:46.631 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:46.631 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:46.631 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:46.631 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.632 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.632 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:46.632 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-17T18:37:46.632 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-17T18:37:46.678 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: res=0.000000 2026-02-17T18:37:46.679 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: echo '0.000000 != 0.000000' 2026-02-17T18:37:46.679 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: bc -l 2026-02-17T18:37:46.680 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:198: TEST_profile_custom_to_builtin: teardown td/mclock-config 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:46.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:46.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:46.787 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:46.788 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:46.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:46.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:46.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:46.790 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:46.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:46.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:46.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:46.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:46.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:46.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:46.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:46.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:46.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:46.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:46.800 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:46.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:46.801 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:46.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:46.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:46.803 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:46.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:46.804 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:46.804 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.804 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:46.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:46.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:46.807 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:46.807 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.807 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.807 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:46.808 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:46.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:46.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:46.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:46.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:46.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:46.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:46.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:46.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:46.812 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:46.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:46.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:46.814 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:46.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:46.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:46.816 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:46.816 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.816 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:46.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:46.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:46.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T18:37:46.818 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:46.818 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.818 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.818 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.18341 2026-02-17T18:37:46.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:46.821 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:37:46.821 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:46.821 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T18:37:46.821 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_recovery_limit_adjustment_mclock td/mclock-config 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: setup td/mclock-config 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:46.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:46.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:46.823 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:46.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:46.824 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:46.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:46.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:46.825 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:46.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:46.826 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:46.826 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:46.827 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:46.827 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:46.828 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:46.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:46.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:46.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:46.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:46.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-17T18:37:46.831 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:46.831 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.831 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.18341 2026-02-17T18:37:46.832 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:46.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:37:46.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:46.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-17T18:37:46.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:205: TEST_recovery_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-17T18:37:46.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-17T18:37:46.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:37:46.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:37:46.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:37:46.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-17T18:37:46.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-17T18:37:46.850 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:37:46.850 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:46.850 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:46.850 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:46.850 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.850 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.850 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:46.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:37:46.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:37:46.875 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:37:46.875 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:46.875 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:46.875 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:37:46.875 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:37:46.875 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:46.876 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:46.876 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:46.877 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:46.877 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.877 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.877 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:46.877 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:46.877 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get fsid 2026-02-17T18:37:46.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:37:46.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:46.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:46.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:37:46.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:37:46.922 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:46.922 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:46.922 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:46.923 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:46.923 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:46.923 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:46.923 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-mon.a.asok 2026-02-17T18:37:46.923 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:46.923 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.18341/ceph-mon.a.asok config get mon_host 2026-02-17T18:37:46.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:206: TEST_recovery_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-17T18:37:46.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-17T18:37:46.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:37:46.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:37:46.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:37:46.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-17T18:37:46.965 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:37:47.064 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:37:47.064 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:47.064 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:47.064 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:47.064 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:47.064 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:47.065 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:47.065 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:47.065 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:47.082 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:208: TEST_recovery_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-17T18:37:47.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:47.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:47.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:37:47.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T18:37:47.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-17T18:37:47.086 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:37:47.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=18da6cc6-66a1-49a1-89dd-31cf1fb13d0f 2026-02-17T18:37:47.087 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 18da6cc6-66a1-49a1-89dd-31cf1fb13d0f 2026-02-17T18:37:47.087 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 18da6cc6-66a1-49a1-89dd-31cf1fb13d0f' 2026-02-17T18:37:47.087 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:37:47.092 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB7tZRpg59nBRAAv8zehOnzIdvrcHkQOSr/Lw== 2026-02-17T18:37:47.092 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB7tZRpg59nBRAAv8zehOnzIdvrcHkQOSr/Lw=="}' 2026-02-17T18:37:47.092 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 18da6cc6-66a1-49a1-89dd-31cf1fb13d0f -i td/mclock-config/0/new.json 2026-02-17T18:37:47.193 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:47.199 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-17T18:37:47.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQB7tZRpg59nBRAAv8zehOnzIdvrcHkQOSr/Lw== --osd-uuid 18da6cc6-66a1-49a1-89dd-31cf1fb13d0f 2026-02-17T18:37:47.216 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.214+0000 7f49f8821780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:47.217 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.216+0000 7f49f8821780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:47.218 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.217+0000 7f49f8821780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:47.219 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.217+0000 7f49f8821780 -1 bdev(0x290cc00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:37:47.219 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.217+0000 7f49f8821780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-17T18:37:47.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-17T18:37:47.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:37:47.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:37:47.656 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:37:47.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:37:47.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:37:47.762 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:37:47.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=5f17cdf5-c1b8-4c5c-ac8d-701a82b7ff4c --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.18341/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T18:37:47.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:37:47.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:37:47.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:37:47.776 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.774+0000 7f75fa233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:47.777 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.776+0000 7f75fa233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:47.778 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:47.776+0000 7f75fa233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:47.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:37:47.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:47.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:37:47.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:47.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:47.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:47.857 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:47.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:47.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:47.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:47.951 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:37:48.017 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:48.015+0000 7f75fa233780 -1 Falling back to public interface 2026-02-17T18:37:48.334 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:48.332+0000 7f75fa233780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:37:48.952 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:37:48.952 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:48.953 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:48.954 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:37:48.954 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:48.954 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:49.050 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/875853237,v1:127.0.0.1:6801/875853237] [v2:127.0.0.1:6802/875853237,v1:127.0.0.1:6803/875853237] exists,up 18da6cc6-66a1-49a1-89dd-31cf1fb13d0f 2026-02-17T18:37:49.051 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:49.051 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:49.051 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:49.052 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-17T18:37:49.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: local 'recoveries={"osd_recovery_max_active":"0"}' 2026-02-17T18:37:49.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: echo '{"osd_recovery_max_active":"0"}' 2026-02-17T18:37:49.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: grep --quiet osd_recovery_max_active 2026-02-17T18:37:49.100 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:217: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-17T18:37:49.328 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:218: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-17T18:37:51.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:51.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:51.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:51.332 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:51.332 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:51.332 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:51.332 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:51.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:51.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-17T18:37:51.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: local 'max_recoveries={"osd_recovery_max_active":"0"}' 2026-02-17T18:37:51.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"0"}' = '{"osd_recovery_max_active":"0"}' 2026-02-17T18:37:51.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:225: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-17T18:37:51.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:226: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-17T18:37:51.832 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:227: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-17T18:37:53.834 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-17T18:37:53.835 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:37:53.835 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:37:53.835 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:53.835 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:53.835 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:53.835 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.18341/ceph-osd.0.asok 2026-02-17T18:37:53.836 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-17T18:37:53.836 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.18341/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-17T18:37:53.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: max_recoveries='{"osd_recovery_max_active":"10"}' 2026-02-17T18:37:53.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:230: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"10"}' = '{"osd_recovery_max_active":"10"}' 2026-02-17T18:37:53.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:232: TEST_recovery_limit_adjustment_mclock: teardown td/mclock-config 2026-02-17T18:37:53.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:53.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:53.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:53.886 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:53.886 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:53.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:53.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:53.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:53.992 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:53.992 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:53.993 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:53.993 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:53.995 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:53.995 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:53.995 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:53.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:53.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:53.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:53.997 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:53.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:53.999 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:53.999 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:54.002 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:54.002 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.002 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:54.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:54.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:54.005 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:54.005 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:54.005 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:54.006 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:54.006 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:54.007 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:54.007 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:54.008 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:54.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:54.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:54.009 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:54.009 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:54.009 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:54.010 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:54.010 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:54.012 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:54.012 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.012 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:54.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:54.013 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:54.013 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/mclock-config 0 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:54.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:54.015 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:54.015 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:54.016 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:54.016 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:54.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:54.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:54.017 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:54.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:54.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:54.019 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:54.019 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:54.019 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:54.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T18:37:54.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-17T18:37:54.021 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:54.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.18341 2026-02-17T18:37:54.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.18341 2026-02-17T18:37:54.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:54.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:54.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T18:37:54.024 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T18:37:54.024 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T18:37:54.089 INFO:tasks.workunit:Running workunit misc/network-ping.sh... 2026-02-17T18:37:54.089 DEBUG:teuthology.orchestra.run.trial068:workunit test misc/network-ping.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh 2026-02-17T18:37:54.151 INFO:tasks.workunit.client.0.trial068.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T18:37:54.155 INFO:tasks.workunit.client.0.trial068.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T18:37:54.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:37:54.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:37:54.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T18:37:54.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T18:37:54.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/network-ping 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:6: run: local dir=td/network-ping 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:7: run: shift 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: export CEPH_MON=127.0.0.1:7146 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: CEPH_MON=127.0.0.1:7146 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:10: run: export CEPH_ARGS 2026-02-17T18:37:54.156 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: uuidgen 2026-02-17T18:37:54.158 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: CEPH_ARGS+='--fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none ' 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:12: run: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:13: run: CEPH_ARGS+='--debug_disable_randomized_ping=true ' 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:14: run: CEPH_ARGS+='--debug_heartbeat_testing_span=5 ' 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:15: run: CEPH_ARGS+='--osd_heartbeat_interval=1 ' 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: set 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: local 'funcs=TEST_network_ping_test1 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:TEST_network_ping_test2' 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-17T18:37:54.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:37:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:37:54.161 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:37:54.161 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:37:54.161 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:37:54.162 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:37:54.162 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:37:54.162 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:37:54.163 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:37:54.163 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:54.163 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:37:54.163 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:37:54.163 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:37:54.163 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:37:54.164 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:37:54.164 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T18:37:54.165 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:37:54.165 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.166 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.28424 2026-02-17T18:37:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:37:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:37:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-17T18:37:54.167 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:37:54.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:54.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.28424 2026-02-17T18:37:54.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test1 td/network-ping 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:25: TEST_network_ping_test1: local dir=td/network-ping 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:27: TEST_network_ping_test1: run_mon td/network-ping a 2026-02-17T18:37:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-17T18:37:54.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:37:54.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:37:54.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:37:54.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-17T18:37:54.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-17T18:37:54.187 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:37:54.187 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:54.188 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:54.188 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:54.188 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.188 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:54.188 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:54.188 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:37:54.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:37:54.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:37:54.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:54.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:54.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:37:54.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:37:54.214 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:54.214 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:54.214 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:54.214 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:54.214 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.214 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:54.214 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mon.a.asok 2026-02-17T18:37:54.215 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:54.215 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.28424/ceph-mon.a.asok config get fsid 2026-02-17T18:37:54.258 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:37:54.258 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mon.a.asok 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:37:54.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.28424/ceph-mon.a.asok config get mon_host 2026-02-17T18:37:54.301 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:28: TEST_network_ping_test1: run_mgr td/network-ping x 2026-02-17T18:37:54.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-17T18:37:54.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:37:54.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:37:54.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:37:54.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-17T18:37:54.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:37:54.416 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:37:54.416 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:54.417 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:54.417 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:54.417 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.417 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:54.417 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:54.417 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:54.418 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:37:54.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:29: TEST_network_ping_test1: run_osd td/network-ping 0 2026-02-17T18:37:54.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T18:37:54.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-17T18:37:54.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:54.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:37:54.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-17T18:37:54.440 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:37:54.440 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f0060d91-b606-4beb-b61f-a570bdd19ef6 2026-02-17T18:37:54.440 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f0060d91-b606-4beb-b61f-a570bdd19ef6' 2026-02-17T18:37:54.440 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 f0060d91-b606-4beb-b61f-a570bdd19ef6 2026-02-17T18:37:54.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:37:54.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCCtZRpU+NzGhAAUtDW8yOng2/TfBVNcGSdbA== 2026-02-17T18:37:54.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCCtZRpU+NzGhAAUtDW8yOng2/TfBVNcGSdbA=="}' 2026-02-17T18:37:54.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f0060d91-b606-4beb-b61f-a570bdd19ef6 -i td/network-ping/0/new.json 2026-02-17T18:37:54.548 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:54.554 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-17T18:37:54.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCCtZRpU+NzGhAAUtDW8yOng2/TfBVNcGSdbA== --osd-uuid f0060d91-b606-4beb-b61f-a570bdd19ef6 2026-02-17T18:37:54.572 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:54.570+0000 7f84c842c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:54.573 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:54.572+0000 7f84c842c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:54.574 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:54.573+0000 7f84c842c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:54.575 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:54.573+0000 7f84c842c780 -1 bdev(0x376ec00 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:37:54.575 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:54.573+0000 7f84c842c780 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-17T18:37:54.946 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-17T18:37:54.947 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:37:54.947 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:37:54.948 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:37:54.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:37:55.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:37:55.049 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:37:55.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:37:55.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:37:55.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:37:55.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:37:55.064 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:55.063+0000 7f9804c25780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:55.065 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:55.064+0000 7f9804c25780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:55.066 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:55.065+0000 7f9804c25780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:55.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:37:55.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:55.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:37:55.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:55.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:55.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:55.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:55.151 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:55.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:55.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:55.246 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:37:55.275 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:55.274+0000 7f9804c25780 -1 Falling back to public interface 2026-02-17T18:37:55.673 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:55.671+0000 7f9804c25780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:37:56.247 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:37:56.247 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:56.247 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:56.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:37:56.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:56.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:37:56.343 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4206053348,v1:127.0.0.1:6801/4206053348] [v2:127.0.0.1:6802/4206053348,v1:127.0.0.1:6803/4206053348] exists,up f0060d91-b606-4beb-b61f-a570bdd19ef6 2026-02-17T18:37:56.347 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:30: TEST_network_ping_test1: run_osd td/network-ping 1 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:56.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:56.349 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:56.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:37:56.351 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:37:56.351 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-17T18:37:56.351 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:37:56.351 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d0f7078e-32d6-4aee-ae4d-9f47b2643044 2026-02-17T18:37:56.351 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d0f7078e-32d6-4aee-ae4d-9f47b2643044' 2026-02-17T18:37:56.351 INFO:tasks.workunit.client.0.trial068.stdout:add osd1 d0f7078e-32d6-4aee-ae4d-9f47b2643044 2026-02-17T18:37:56.351 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:37:56.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEtZRpBzQ4FRAAH0L9rBeqKeECra58e0CTMw== 2026-02-17T18:37:56.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEtZRpBzQ4FRAAH0L9rBeqKeECra58e0CTMw=="}' 2026-02-17T18:37:56.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d0f7078e-32d6-4aee-ae4d-9f47b2643044 -i td/network-ping/1/new.json 2026-02-17T18:37:56.460 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:56.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-17T18:37:56.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEtZRpBzQ4FRAAH0L9rBeqKeECra58e0CTMw== --osd-uuid d0f7078e-32d6-4aee-ae4d-9f47b2643044 2026-02-17T18:37:56.483 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:56.481+0000 7f6db0629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:56.484 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:56.483+0000 7f6db0629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:56.485 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:56.484+0000 7f6db0629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:56.486 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:56.484+0000 7f6db0629780 -1 bdev(0x37b0c00 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-17T18:37:56.486 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:56.484+0000 7f6db0629780 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-17T18:37:56.808 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-17T18:37:56.808 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:37:56.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T18:37:56.809 INFO:tasks.workunit.client.0.trial068.stdout:adding osd1 key to auth repository 2026-02-17T18:37:56.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:37:57.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T18:37:57.119 INFO:tasks.workunit.client.0.trial068.stdout:start osd.1 2026-02-17T18:37:57.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:37:57.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:37:57.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:37:57.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:37:57.135 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:57.133+0000 7f325a22f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:57.136 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:57.135+0000 7f325a22f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:57.137 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:57.135+0000 7f325a22f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:57.267 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:57.265+0000 7f325a22f780 -1 Falling back to public interface 2026-02-17T18:37:57.342 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T18:37:57.342 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:57.342 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T18:37:57.342 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:57.342 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:57.342 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:57.342 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:57.343 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:57.343 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:57.343 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:37:57.565 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:57.563+0000 7f325a22f780 -1 osd.1 0 log_to_monitors true 2026-02-17T18:37:57.566 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:37:58.567 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:37:58.567 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:58.568 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:37:58.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:37:58.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:58.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:37:58.791 INFO:tasks.workunit.client.0.trial068.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3880791954,v1:127.0.0.1:6811/3880791954] [v2:127.0.0.1:6812/3880791954,v1:127.0.0.1:6813/3880791954] exists,up d0f7078e-32d6-4aee-ae4d-9f47b2643044 2026-02-17T18:37:58.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:37:58.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:37:58.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:37:58.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:31: TEST_network_ping_test1: run_osd td/network-ping 2 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-17T18:37:58.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:37:58.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b5801400-4d64-4b3c-9b09-8d3739e303bb 2026-02-17T18:37:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b5801400-4d64-4b3c-9b09-8d3739e303bb' 2026-02-17T18:37:58.799 INFO:tasks.workunit.client.0.trial068.stdout:add osd2 b5801400-4d64-4b3c-9b09-8d3739e303bb 2026-02-17T18:37:58.799 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:37:58.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCGtZRp8PPuLxAA+74Ll5GbyqHMct1lYnT+WQ== 2026-02-17T18:37:58.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCGtZRp8PPuLxAA+74Ll5GbyqHMct1lYnT+WQ=="}' 2026-02-17T18:37:58.805 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b5801400-4d64-4b3c-9b09-8d3739e303bb -i td/network-ping/2/new.json 2026-02-17T18:37:59.030 INFO:tasks.workunit.client.0.trial068.stdout:2 2026-02-17T18:37:59.037 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-17T18:37:59.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGtZRp8PPuLxAA+74Ll5GbyqHMct1lYnT+WQ== --osd-uuid b5801400-4d64-4b3c-9b09-8d3739e303bb 2026-02-17T18:37:59.054 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.053+0000 7f23f8a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:59.055 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.054+0000 7f23f8a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:59.056 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.055+0000 7f23f8a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:59.057 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.056+0000 7f23f8a65780 -1 bdev(0x363ac00 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-17T18:37:59.057 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.056+0000 7f23f8a65780 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-17T18:37:59.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-17T18:37:59.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:37:59.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T18:37:59.382 INFO:tasks.workunit.client.0.trial068.stdout:adding osd2 key to auth repository 2026-02-17T18:37:59.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:37:59.701 INFO:tasks.workunit.client.0.trial068.stdout:start osd.2 2026-02-17T18:37:59.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T18:37:59.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:37:59.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:37:59.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:37:59.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:37:59.716 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.715+0000 7f37a0a62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:59.717 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.716+0000 7f37a0a62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:59.718 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.717+0000 7f37a0a62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:37:59.853 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:37:59.852+0000 7f37a0a62780 -1 Falling back to public interface 2026-02-17T18:37:59.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T18:37:59.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:37:59.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T18:37:59.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:37:59.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:37:59.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:37:59.926 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:37:59.927 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:37:59.927 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:37:59.927 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:38:00.151 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:38:00.150+0000 7f37a0a62780 -1 osd.2 0 log_to_monitors true 2026-02-17T18:38:00.153 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:38:01.154 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:38:01.154 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:38:01.155 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:38:01.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:38:01.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:38:01.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:38:01.380 INFO:tasks.workunit.client.0.trial068.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3172156044,v1:127.0.0.1:6819/3172156044] [v2:127.0.0.1:6820/3172156044,v1:127.0.0.1:6821/3172156044] exists,up b5801400-4d64-4b3c-9b09-8d3739e303bb 2026-02-17T18:38:01.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:38:01.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:38:01.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:38:01.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:33: TEST_network_ping_test1: sleep 5 2026-02-17T18:38:06.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:35: TEST_network_ping_test1: create_pool foo 16 2026-02-17T18:38:06.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-17T18:38:06.649 INFO:tasks.workunit.client.0.trial068.stderr:pool 'foo' already exists 2026-02-17T18:38:06.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:38:07.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:38: TEST_network_ping_test1: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-17T18:38:07.679 INFO:tasks.workunit.client.0.trial068.stdout:hints = 1 2026-02-17T18:38:07.679 INFO:tasks.workunit.client.0.trial068.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-17T18:38:07.679 INFO:tasks.workunit.client.0.trial068.stdout:Object prefix: benchmark_data_trial068_33085 2026-02-17T18:38:07.679 INFO:tasks.workunit.client.0.trial068.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T18:38:07.679 INFO:tasks.workunit.client.0.trial068.stdout: 0 0 0 0 0 0 - 0 2026-02-17T18:38:08.679 INFO:tasks.workunit.client.0.trial068.stdout: 1 16 11294 11278 44.0509 44.0547 0.00146587 0.00141696 2026-02-17T18:38:09.679 INFO:tasks.workunit.client.0.trial068.stdout: 2 16 23221 23205 45.3186 46.5898 0.00108474 0.00137743 2026-02-17T18:38:10.679 INFO:tasks.workunit.client.0.trial068.stdout: 3 16 35317 35301 45.9611 47.25 0.00194867 0.00135822 2026-02-17T18:38:11.679 INFO:tasks.workunit.client.0.trial068.stdout: 4 16 47843 47827 46.7024 48.9297 0.0012933 0.00133684 2026-02-17T18:38:12.679 INFO:tasks.workunit.client.0.trial068.stdout: 5 16 59637 59621 46.5753 46.0703 0.000542695 0.00133995 2026-02-17T18:38:13.679 INFO:tasks.workunit.client.0.trial068.stdout: 6 16 71290 71274 46.3988 45.5195 0.00137149 0.00134568 2026-02-17T18:38:14.679 INFO:tasks.workunit.client.0.trial068.stdout: 7 16 83579 83563 46.6275 48.0039 0.0015042 0.00133909 2026-02-17T18:38:15.679 INFO:tasks.workunit.client.0.trial068.stdout: 8 16 94146 94130 45.9584 41.2773 0.00168862 0.00135859 2026-02-17T18:38:16.679 INFO:tasks.workunit.client.0.trial068.stdout: 9 16 104723 104707 45.4422 41.3164 0.00173333 0.00137409 2026-02-17T18:38:17.679 INFO:tasks.workunit.client.0.trial068.stdout: 10 3 115110 115107 44.9602 40.625 0.00162121 0.00138889 2026-02-17T18:38:18.680 INFO:tasks.workunit.client.0.trial068.stdout:Total time run: 10.001 2026-02-17T18:38:18.680 INFO:tasks.workunit.client.0.trial068.stdout:Total writes made: 115110 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Write size: 4096 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Object size: 4096 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Bandwidth (MB/sec): 44.9605 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Stddev Bandwidth: 2.99851 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Max bandwidth (MB/sec): 48.9297 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Min bandwidth (MB/sec): 40.625 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Average IOPS: 11509 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Stddev IOPS: 767.619 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Max IOPS: 12526 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Min IOPS: 10400 2026-02-17T18:38:18.681 INFO:tasks.workunit.client.0.trial068.stdout:Average Latency(s): 0.00138887 2026-02-17T18:38:18.682 INFO:tasks.workunit.client.0.trial068.stdout:Stddev Latency(s): 0.000719631 2026-02-17T18:38:18.682 INFO:tasks.workunit.client.0.trial068.stdout:Max latency(s): 0.0148264 2026-02-17T18:38:18.682 INFO:tasks.workunit.client.0.trial068.stdout:Min latency(s): 0.000366816 2026-02-17T18:38:18.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:41: TEST_network_ping_test1: sleep 10 2026-02-17T18:38:28.686 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:42: TEST_network_ping_test1: flush_pg_stats 2026-02-17T18:38:28.686 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:38:28.687 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:38:28.910 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:38:28.910 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:38:28.911 INFO:tasks.workunit.client.0.trial068.stderr:2' 2026-02-17T18:38:28.911 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:38:28.911 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:38:28.911 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:38:28.976 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-17T18:38:28.976 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-17T18:38:28.976 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T18:38:28.976 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:38:28.976 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:38:29.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-17T18:38:29.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-17T18:38:29.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489 1-38654705674' 2026-02-17T18:38:29.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:38:29.042 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:38:29.106 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-17T18:38:29.106 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-17T18:38:29.106 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489 1-38654705674 2-55834574856' 2026-02-17T18:38:29.106 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:38:29.107 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836489 2026-02-17T18:38:29.107 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:38:29.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:38:29.108 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-17T18:38:29.108 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:38:29.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836489 2026-02-17T18:38:29.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T18:38:29.109 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836489 2026-02-17T18:38:29.110 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:38:29.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-17T18:38:29.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:38:30.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:38:30.337 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:38:30.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T18:38:30.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:38:30.568 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705674 2026-02-17T18:38:30.569 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:38:30.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:38:30.570 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-17T18:38:30.570 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:38:30.571 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705674 2026-02-17T18:38:30.572 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705674 2026-02-17T18:38:30.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T18:38:30.572 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:38:30.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T18:38:30.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:38:30.796 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574856 2026-02-17T18:38:30.796 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:38:30.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:38:30.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574856 2026-02-17T18:38:30.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:38:30.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-17T18:38:30.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574856' 2026-02-17T18:38:30.799 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 55834574856 2026-02-17T18:38:30.799 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:38:31.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:31.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-osd.0.asok 2026-02-17T18:38:31.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:31.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-osd.0.asok dump_osd_network 2026-02-17T18:38:31.064 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:31.064 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 1000, 2026-02-17T18:38:31.064 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:38:31.064 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:31.068 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.068 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:31.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:31.071 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.071 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:31.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T18:38:31.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mgr.x.asok 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:31.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-mgr.x.asok dump_osd_network 2026-02-17T18:38:31.112 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:31.112 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 1000, 2026-02-17T18:38:31.112 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:38:31.113 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:31.117 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.117 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:31.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:31.120 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.120 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:31.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T18:38:31.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:31.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-osd.0.asok 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:31.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-osd.0.asok dump_osd_network 0 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 0, 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [ 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:29 2026", 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.167 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.123, 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.166, 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.158 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.156, 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.450, 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.450 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.156, 2026-02-17T18:38:31.168 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.450, 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.450 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.156 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.137, 2026-02-17T18:38:31.169 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.144, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.160 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.167, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.225, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.225 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.167, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.225, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.225 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.170 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.176 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.119, 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.143, 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.158 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.171 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.131, 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.244, 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.244 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.131, 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.244, 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.244 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.166 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.172 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:29 2026", 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.095, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.134, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.125 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.124, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.419, 2026-02-17T18:38:31.173 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.419 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.124, 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.419, 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.419 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.131 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: } 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout: ] 2026-02-17T18:38:31.174 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:31.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:31.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: test 4 = 4 2026-02-17T18:38:31.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:31.177 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:31.177 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:31.178 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T18:38:31.178 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T18:38:31.178 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T18:38:31.179 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:31.179 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:31.179 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:31.179 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mgr.x.asok 2026-02-17T18:38:31.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:31.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-mgr.x.asok dump_osd_network 0 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 0, 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [ 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:28 2026", 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.217 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.116, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.231, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.587 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.106, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.086, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.086 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.135, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 1.479, 2026-02-17T18:38:31.218 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 2.599 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.113 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:28 2026", 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.102, 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.159, 2026-02-17T18:38:31.219 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.273 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.088, 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.088, 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.088 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.123, 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.919, 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 1.242 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.098 2026-02-17T18:38:31.220 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:27 2026", 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.135, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.173, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.151 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.105, 2026-02-17T18:38:31.221 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.096, 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.096 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.160, 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.392, 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.392 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.132 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:24 2026", 2026-02-17T18:38:31.222 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.114, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.173, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.160 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.103, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.103, 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.103 2026-02-17T18:38:31.223 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.128, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.450, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.450 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.116 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:31.224 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.132, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.167, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.154 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.106, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.104, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.104 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.172, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.376, 2026-02-17T18:38:31.225 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.376 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.108 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.137, 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.144, 2026-02-17T18:38:31.226 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.160 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.094, 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.087, 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.087 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.167, 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.225, 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.225 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.128 2026-02-17T18:38:31.227 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.119, 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.143, 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.158 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.228 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.105, 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.105, 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.105 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.131, 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.244, 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.244 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.142 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:27 2026", 2026-02-17T18:38:31.229 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.119, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.153, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.149 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.097, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.097, 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.097 2026-02-17T18:38:31.230 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.148, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.243, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.243 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.110 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.231 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.113, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.152, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.147 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.091, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.091, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.091 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.160, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.360, 2026-02-17T18:38:31.232 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.360 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.076 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.111, 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.137, 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.147 2026-02-17T18:38:31.233 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.096, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.089, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.089 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.140, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.247, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.247 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.098 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.234 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:26 2026", 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.092, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.120, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.140 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.072, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.072, 2026-02-17T18:38:31.235 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.072 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.122, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.351, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.351 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.086 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:24 2026", 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:31.236 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.112, 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.140, 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.127 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.088, 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.088, 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.088 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.151, 2026-02-17T18:38:31.237 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.419, 2026-02-17T18:38:31.238 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.419 2026-02-17T18:38:31.238 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:31.238 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.103 2026-02-17T18:38:31.238 INFO:tasks.workunit.client.0.trial068.stdout: } 2026-02-17T18:38:31.238 INFO:tasks.workunit.client.0.trial068.stdout: ] 2026-02-17T18:38:31.238 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:31.239 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.239 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:31.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: test 12 = 12 2026-02-17T18:38:31.239 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:31.239 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:31.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:31.240 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:61: TEST_network_ping_test1: sleep 20 2026-02-17T18:38:51.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:62: TEST_network_ping_test1: flush_pg_stats 2026-02-17T18:38:51.238 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:38:51.238 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:38:51.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:38:51.465 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:38:51.465 INFO:tasks.workunit.client.0.trial068.stderr:2' 2026-02-17T18:38:51.465 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:38:51.465 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:38:51.465 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:38:51.530 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836496 2026-02-17T18:38:51.530 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836496 2026-02-17T18:38:51.530 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496' 2026-02-17T18:38:51.530 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:38:51.530 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:38:51.595 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705680 2026-02-17T18:38:51.595 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705680 2026-02-17T18:38:51.595 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496 1-38654705680' 2026-02-17T18:38:51.595 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:38:51.595 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:38:51.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-17T18:38:51.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-17T18:38:51.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496 1-38654705680 2-55834574863' 2026-02-17T18:38:51.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:38:51.661 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836496 2026-02-17T18:38:51.662 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:38:51.662 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:38:51.663 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836496 2026-02-17T18:38:51.663 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:38:51.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836496 2026-02-17T18:38:51.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-17T18:38:51.664 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836496 2026-02-17T18:38:51.665 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:38:51.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836493 -lt 21474836496 2026-02-17T18:38:51.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:38:52.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:38:52.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:38:53.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-17T18:38:53.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:38:53.116 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705680 2026-02-17T18:38:53.116 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:38:53.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:38:53.117 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705680 2026-02-17T18:38:53.117 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:38:53.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705680 2026-02-17T18:38:53.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-17T18:38:53.119 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705680 2026-02-17T18:38:53.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:38:53.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-17T18:38:53.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:38:53.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574863 2026-02-17T18:38:53.351 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:38:53.351 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:38:53.352 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574863 2026-02-17T18:38:53.352 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:38:53.353 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574863 2026-02-17T18:38:53.353 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-17T18:38:53.354 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 55834574863 2026-02-17T18:38:53.354 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:38:53.577 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-17T18:38:53.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:53.579 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T18:38:53.579 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:38:53.579 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:38:53.579 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:53.579 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:53.580 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:53.580 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-osd.0.asok 2026-02-17T18:38:53.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:53.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-osd.0.asok dump_osd_network 2026-02-17T18:38:53.623 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:53.624 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 1000, 2026-02-17T18:38:53.624 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:38:53.624 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:53.630 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.630 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:53.632 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:53.633 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.633 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:53.634 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T18:38:53.635 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:53.636 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T18:38:53.636 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T18:38:53.636 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T18:38:53.636 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:53.636 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:53.637 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:53.637 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mgr.x.asok 2026-02-17T18:38:53.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:53.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-mgr.x.asok dump_osd_network 2026-02-17T18:38:53.673 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:53.673 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 1000, 2026-02-17T18:38:53.673 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:38:53.673 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:53.677 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.677 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:53.680 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:53.680 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.681 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:53.682 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T18:38:53.683 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:53.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T18:38:53.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:38:53.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:38:53.685 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:53.685 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:53.685 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:53.685 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-osd.0.asok 2026-02-17T18:38:53.685 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:53.685 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-osd.0.asok dump_osd_network 0 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 0, 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [ 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:51 2026", 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.727 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.153, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.123, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.144 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.218, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.218, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.244 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.218, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.218, 2026-02-17T18:38:53.728 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.244 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.184 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:49 2026", 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.124, 2026-02-17T18:38:53.729 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.121, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.149 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.200, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.200, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.450 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.200, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.200, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.450 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.730 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.161 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:51 2026", 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.137, 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.118, 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.143 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.731 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.188, 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.188, 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.225 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.188, 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.188, 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.225 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.174 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.732 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:49 2026", 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.077, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.090, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.117 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.092, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.131, 2026-02-17T18:38:53.733 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.419 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.092, 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.131, 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.419 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.149 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: } 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout: ] 2026-02-17T18:38:53.734 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:53.735 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.735 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:53.735 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: test 4 = 4 2026-02-17T18:38:53.735 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.736 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:53.736 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:53.736 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:38:53.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mgr.x.asok 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:38:53.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-mgr.x.asok dump_osd_network 0 2026-02-17T18:38:53.774 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 0, 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [ 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:48 2026", 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.114, 2026-02-17T18:38:53.775 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.118, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.415 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.104, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.084, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.084 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.128, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.196, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 2.599 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.776 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.165 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:48 2026", 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.093, 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.123, 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.220 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.777 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.084, 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.073, 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.073 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.106, 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.275, 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 1.242 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.139 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.778 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:51 2026", 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.138, 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.143, 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.152 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.084, 2026-02-17T18:38:53.779 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.084, 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.084 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.167, 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.321, 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.376 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.084 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:47 2026", 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.780 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.105, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.141, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.152 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.092, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.092, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.092 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.781 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.138, 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.302, 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.392 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.149 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:47 2026", 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.782 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.106, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.141, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.149 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.095, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.095, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.095 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.131, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.320, 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.320 2026-02-17T18:38:53.783 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.157 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:49 2026", 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.124, 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.121, 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.149 2026-02-17T18:38:53.784 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.100, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.091, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.091 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.200, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.200, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.450 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.080 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.785 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:46 2026", 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.089, 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.118, 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.146 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.080, 2026-02-17T18:38:53.786 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.080, 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.080 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.100, 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.176, 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.225 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.089 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:46 2026", 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.787 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.085, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.116, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.145 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.073, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.073, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.073 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.788 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.111, 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.179, 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.244 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.122 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:51 2026", 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.789 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.126, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.134, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.143 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.077, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.076, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.076 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.194, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.312, 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.360 2026-02-17T18:38:53.790 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.077 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:51 2026", 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.130, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.116, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.135 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.791 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.096, 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.070, 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.070 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.178, 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.181, 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.247 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.096 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.792 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:51 2026", 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.103, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.105, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.125 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.072, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.072, 2026-02-17T18:38:53.793 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.072 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.150, 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.191, 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.351 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.072 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:38:49 2026", 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:38:53.794 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.077, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.090, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.117 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.063, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.062, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.062 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.092, 2026-02-17T18:38:53.795 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.131, 2026-02-17T18:38:53.796 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.419 2026-02-17T18:38:53.796 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:38:53.796 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.071 2026-02-17T18:38:53.796 INFO:tasks.workunit.client.0.trial068.stdout: } 2026-02-17T18:38:53.796 INFO:tasks.workunit.client.0.trial068.stdout: ] 2026-02-17T18:38:53.796 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:38:53.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:38:53.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: test 12 = 12 2026-02-17T18:38:53.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:38:53.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: jq .threshold 2026-02-17T18:38:53.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:38:53.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:81: TEST_network_ping_test1: sleep 50 2026-02-17T18:39:43.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:82: TEST_network_ping_test1: flush_pg_stats 2026-02-17T18:39:43.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:39:43.786 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:39:44.013 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:39:44.013 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:39:44.013 INFO:tasks.workunit.client.0.trial068.stderr:2' 2026-02-17T18:39:44.013 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:39:44.013 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:39:44.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:39:44.080 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836508 2026-02-17T18:39:44.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836508 2026-02-17T18:39:44.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508' 2026-02-17T18:39:44.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:39:44.081 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:39:44.144 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705693 2026-02-17T18:39:44.144 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705693 2026-02-17T18:39:44.145 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705693' 2026-02-17T18:39:44.145 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:39:44.145 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:39:44.209 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574875 2026-02-17T18:39:44.209 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574875 2026-02-17T18:39:44.209 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705693 2-55834574875' 2026-02-17T18:39:44.209 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:39:44.210 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836508 2026-02-17T18:39:44.210 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:39:44.210 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:39:44.211 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836508 2026-02-17T18:39:44.211 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:39:44.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836508 2026-02-17T18:39:44.212 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836508 2026-02-17T18:39:44.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-17T18:39:44.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:39:44.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836506 -lt 21474836508 2026-02-17T18:39:44.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:39:45.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:39:45.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:39:45.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-17T18:39:45.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:39:45.665 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705693 2026-02-17T18:39:45.666 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:39:45.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:39:45.667 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705693 2026-02-17T18:39:45.667 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:39:45.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705693 2026-02-17T18:39:45.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705693' 2026-02-17T18:39:45.668 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705693 2026-02-17T18:39:45.669 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:39:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705693 -lt 38654705693 2026-02-17T18:39:45.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:39:45.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574875 2026-02-17T18:39:45.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:39:45.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:39:45.896 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574875 2026-02-17T18:39:45.896 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:39:45.897 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574875 2026-02-17T18:39:45.897 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574875' 2026-02-17T18:39:45.897 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 55834574875 2026-02-17T18:39:45.898 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:39:46.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574875 -lt 55834574875 2026-02-17T18:39:46.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:39:46.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T18:39:46.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:39:46.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:39:46.132 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:46.132 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:46.132 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:46.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-osd.0.asok 2026-02-17T18:39:46.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:39:46.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-osd.0.asok dump_osd_network 2026-02-17T18:39:46.175 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:39:46.175 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 1000, 2026-02-17T18:39:46.175 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:39:46.175 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:39:46.179 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.179 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:39:46.181 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:39:46.182 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.182 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: jq .threshold 2026-02-17T18:39:46.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T18:39:46.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mgr.x.asok 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:39:46.186 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-mgr.x.asok dump_osd_network 2026-02-17T18:39:46.223 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:39:46.223 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 1000, 2026-02-17T18:39:46.223 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:39:46.223 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:39:46.228 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.228 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:39:46.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:39:46.230 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.231 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: jq .threshold 2026-02-17T18:39:46.232 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: test 1000 = 1000 2026-02-17T18:39:46.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:39:46.234 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T18:39:46.234 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:39:46.234 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:39:46.234 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:46.234 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:46.234 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:46.235 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-osd.0.asok 2026-02-17T18:39:46.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:39:46.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-osd.0.asok dump_osd_network 0 2026-02-17T18:39:46.275 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 0, 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [ 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.129, 2026-02-17T18:39:46.276 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.109, 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.118 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.164, 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.164, 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.227 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.164, 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.164, 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.227 2026-02-17T18:39:46.277 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.121 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:44 2026", 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.125, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.099, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.112 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.278 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.183, 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.183, 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.200 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.183, 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.183, 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.200 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.136 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.279 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.115, 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.101, 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.118 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.133, 2026-02-17T18:39:46.280 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.140, 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.218 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.133, 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.140, 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.218 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.157 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:44 2026", 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.281 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.087, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.082, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.081 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.122, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.146, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.149 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.282 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.122, 2026-02-17T18:39:46.283 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.146, 2026-02-17T18:39:46.283 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.149 2026-02-17T18:39:46.283 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.283 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.107 2026-02-17T18:39:46.283 INFO:tasks.workunit.client.0.trial068.stdout: } 2026-02-17T18:39:46.283 INFO:tasks.workunit.client.0.trial068.stdout: ] 2026-02-17T18:39:46.283 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:39:46.284 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.284 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:39:46.284 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: test 4 = 4 2026-02-17T18:39:46.284 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.284 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: jq .threshold 2026-02-17T18:39:46.285 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:39:46.285 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:39:46.286 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T18:39:46.286 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T18:39:46.287 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T18:39:46.287 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:46.287 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:46.287 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:46.287 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mgr.x.asok 2026-02-17T18:39:46.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:39:46.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-mgr.x.asok dump_osd_network 0 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 0, 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [ 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:39:46.324 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.125, 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.124, 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.135 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.076, 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.076, 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.076 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.325 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.152, 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.248, 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.321 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.154 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.326 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.106, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.118, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.129 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.094, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.094, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.076 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.123, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.200, 2026-02-17T18:39:46.327 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.312 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.175 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.129, 2026-02-17T18:39:46.328 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.109, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.118 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.111, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.074, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.074 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.164, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.164, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.227 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.329 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.109 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:42 2026", 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.121, 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.114, 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.127 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.330 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.108, 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.075, 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.075 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.131, 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.142, 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.302 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.086 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.331 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:42 2026", 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.101, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.109, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.123 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.089, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.087, 2026-02-17T18:39:46.332 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.087 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.122, 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.138, 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.320 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.109 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.333 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 2, 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.115, 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.101, 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.118 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.103, 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.077, 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.073 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.334 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.133, 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.140, 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.218 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.100 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.335 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.097, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.097, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.113 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.080, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.064, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.064 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.114, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.190, 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.190 2026-02-17T18:39:46.336 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.164 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:39 2026", 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.106, 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.093, 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.112 2026-02-17T18:39:46.337 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.091, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.068, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.068 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.116, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.123, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.200 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.117 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.338 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:43 2026", 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "front", 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.093, 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.099, 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.111 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.075, 2026-02-17T18:39:46.339 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.075, 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.075 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.113, 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.140, 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.196 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.075 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:43 2026", 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.340 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 1, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.094, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.090, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.108 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.076, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.067, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.067 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.341 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.116, 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.127, 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.275 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.093 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:41 2026", 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 2, 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 0, 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.342 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.076, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.086, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.103 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.061, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.061, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.061 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.106, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.180, 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.262 2026-02-17T18:39:46.343 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.187 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: { 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "last update": "Tue Feb 17 18:39:39 2026", 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "stale": false, 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "from osd": 0, 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "to osd": 1, 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "interface": "back", 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "average": { 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.098, 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.078, 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.082 2026-02-17T18:39:46.344 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "min": { 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.078, 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.055, 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.055 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "max": { 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "1min": 0.146, 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "5min": 0.146, 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "15min": 0.149 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: }, 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: "last": 0.075 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: } 2026-02-17T18:39:46.345 INFO:tasks.workunit.client.0.trial068.stdout: ] 2026-02-17T18:39:46.346 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:39:46.347 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: jq '.entries | length' 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: test 12 = 12 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: jq .threshold 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: test 0 = 0 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:46.348 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:46.349 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mgr.x.asok 2026-02-17T18:39:46.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:39:46.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-mgr.x.asok dump_osd_network 99 2026-02-17T18:39:46.372 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:39:46.372 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 99, 2026-02-17T18:39:46.372 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:39:46.372 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:39:46.377 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.377 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: jq .threshold 2026-02-17T18:39:46.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: test 99 = 99 2026-02-17T18:39:46.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: tee td/network-ping/json 2026-02-17T18:39:46.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: get_asok_path osd.0 2026-02-17T18:39:46.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:39:46.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:39:46.380 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:46.380 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:46.380 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:46.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-osd.0.asok 2026-02-17T18:39:46.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: CEPH_ARGS= 2026-02-17T18:39:46.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.28424/ceph-osd.0.asok dump_osd_network 98 2026-02-17T18:39:46.422 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:39:46.422 INFO:tasks.workunit.client.0.trial068.stdout: "threshold": 98, 2026-02-17T18:39:46.422 INFO:tasks.workunit.client.0.trial068.stdout: "entries": [] 2026-02-17T18:39:46.423 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:39:46.427 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: cat td/network-ping/json 2026-02-17T18:39:46.427 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: jq .threshold 2026-02-17T18:39:46.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: test 98 = 98 2026-02-17T18:39:46.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:105: TEST_network_ping_test1: rm -f td/network-ping/json 2026-02-17T18:39:46.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-17T18:39:46.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T18:39:46.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:39:46.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T18:39:46.431 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:39:46.431 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:39:46.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:39:46.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:39:46.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:39:46.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:39:46.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:39:46.538 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:39:46.539 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:39:46.540 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:39:46.540 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:39:46.540 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:39:46.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:39:46.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:39:46.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:39:46.541 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:39:46.542 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:39:46.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:39:46.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T18:39:47.167 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:39:47.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.28424 2026-02-17T18:39:47.169 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:39:47.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:39:47.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:39:47.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:39:47.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:39:47.171 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:39:47.172 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:39:47.172 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:39:47.173 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:39:47.173 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:39:47.173 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:39:47.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:39:47.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:39:47.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:39:47.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:39:47.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:39:47.176 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:39:47.176 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T18:39:47.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:39:47.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.177 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.28424 2026-02-17T18:39:47.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:39:47.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:39:47.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-17T18:39:47.179 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:39:47.179 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.179 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.180 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.28424 2026-02-17T18:39:47.180 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:39:47.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:39:47.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:39:47.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-17T18:39:47.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test2 td/network-ping 2026-02-17T18:39:47.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:111: TEST_network_ping_test2: local dir=td/network-ping 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:113: TEST_network_ping_test2: export CEPH_ARGS 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: export 'EXTRA_OPTS= --mon_warn_on_slow_ping_time=0.001' 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: EXTRA_OPTS=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:115: TEST_network_ping_test2: run_mon td/network-ping a 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-17T18:39:47.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-17T18:39:47.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:39:47.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:39:47.201 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:39:47.201 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:39:47.201 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.201 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.201 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:47.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --mon_warn_on_slow_ping_time=0.001 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:39:47.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:39:47.226 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:39:47.226 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:39:47.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:39:47.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:39:47.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:39:47.228 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:39:47.228 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:39:47.228 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:39:47.228 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:47.228 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.228 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.228 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mon.a.asok 2026-02-17T18:39:47.229 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:39:47.229 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.28424/ceph-mon.a.asok config get fsid 2026-02-17T18:39:47.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:39:47.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:39:47.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:39:47.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:39:47.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:39:47.270 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:39:47.270 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:39:47.271 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:39:47.271 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:39:47.272 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.272 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.272 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.28424/ceph-mon.a.asok 2026-02-17T18:39:47.272 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:39:47.272 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.28424/ceph-mon.a.asok config get mon_host 2026-02-17T18:39:47.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:116: TEST_network_ping_test2: run_mgr td/network-ping x 2026-02-17T18:39:47.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-17T18:39:47.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:39:47.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:39:47.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:39:47.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-17T18:39:47.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:39:47.414 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:39:47.414 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:39:47.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:39:47.415 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:39:47.415 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.415 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:47.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:39:47.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --mon_warn_on_slow_ping_time=0.001 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:117: TEST_network_ping_test2: run_osd td/network-ping 0 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:39:47.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:47.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-17T18:39:47.437 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:39:47.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9c8462eb-7848-439f-90dd-35048571df74 2026-02-17T18:39:47.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9c8462eb-7848-439f-90dd-35048571df74' 2026-02-17T18:39:47.438 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 9c8462eb-7848-439f-90dd-35048571df74 2026-02-17T18:39:47.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:39:47.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDztZRpNBpvGhAAlF7eL9zAj5DrnI2y/cRcdQ== 2026-02-17T18:39:47.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDztZRpNBpvGhAAlF7eL9zAj5DrnI2y/cRcdQ=="}' 2026-02-17T18:39:47.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9c8462eb-7848-439f-90dd-35048571df74 -i td/network-ping/0/new.json 2026-02-17T18:39:47.546 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:39:47.553 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-17T18:39:47.554 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDztZRpNBpvGhAAlF7eL9zAj5DrnI2y/cRcdQ== --osd-uuid 9c8462eb-7848-439f-90dd-35048571df74 2026-02-17T18:39:47.570 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:47.569+0000 7f0100e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:47.572 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:47.571+0000 7f0100e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:47.573 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:47.571+0000 7f0100e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:47.573 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:47.572+0000 7f0100e0d780 -1 bdev(0x31e2c00 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:39:47.573 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:47.572+0000 7f0100e0d780 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-17T18:39:47.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-17T18:39:47.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:39:47.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:39:47.982 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:39:47.982 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:39:48.087 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:39:48.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:39:48.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:39:48.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:39:48.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:39:48.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:39:48.103 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:48.101+0000 7fb76a239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:48.104 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:48.103+0000 7fb76a239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:48.105 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:48.104+0000 7fb76a239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:48.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:39:48.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:39:48.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:39:48.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:39:48.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:39:48.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:39:48.188 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:39:48.188 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:39:48.188 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:39:48.188 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:39:48.284 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:39:48.322 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:48.321+0000 7fb76a239780 -1 Falling back to public interface 2026-02-17T18:39:48.636 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:48.635+0000 7fb76a239780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:39:49.286 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:39:49.286 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:39:49.286 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:39:49.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:39:49.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:39:49.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:39:49.384 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2037899264,v1:127.0.0.1:6801/2037899264] [v2:127.0.0.1:6802/2037899264,v1:127.0.0.1:6803/2037899264] exists,up 9c8462eb-7848-439f-90dd-35048571df74 2026-02-17T18:39:49.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:39:49.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:39:49.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:39:49.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:118: TEST_network_ping_test2: run_osd td/network-ping 1 2026-02-17T18:39:49.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T18:39:49.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:39:49.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T18:39:49.389 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T18:39:49.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b862fb4c-fd2a-41b3-9035-b2552cd9189f 2026-02-17T18:39:49.391 INFO:tasks.workunit.client.0.trial068.stdout:add osd1 b862fb4c-fd2a-41b3-9035-b2552cd9189f 2026-02-17T18:39:49.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b862fb4c-fd2a-41b3-9035-b2552cd9189f' 2026-02-17T18:39:49.392 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:39:49.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD1tZRpTM20FxAA6HGfmLQOcaBPpOiW1QqtYA== 2026-02-17T18:39:49.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD1tZRpTM20FxAA6HGfmLQOcaBPpOiW1QqtYA=="}' 2026-02-17T18:39:49.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b862fb4c-fd2a-41b3-9035-b2552cd9189f -i td/network-ping/1/new.json 2026-02-17T18:39:49.502 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:39:49.508 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-17T18:39:49.509 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD1tZRpTM20FxAA6HGfmLQOcaBPpOiW1QqtYA== --osd-uuid b862fb4c-fd2a-41b3-9035-b2552cd9189f 2026-02-17T18:39:49.525 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:49.524+0000 7ff1ca057780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:49.526 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:49.525+0000 7ff1ca057780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:49.527 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:49.526+0000 7ff1ca057780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:49.527 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:49.526+0000 7ff1ca057780 -1 bdev(0x32fcc00 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-17T18:39:49.528 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:49.526+0000 7ff1ca057780 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-17T18:39:49.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-17T18:39:49.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:39:49.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T18:39:49.854 INFO:tasks.workunit.client.0.trial068.stdout:adding osd1 key to auth repository 2026-02-17T18:39:49.854 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:39:50.183 INFO:tasks.workunit.client.0.trial068.stdout:start osd.1 2026-02-17T18:39:50.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T18:39:50.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:39:50.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:39:50.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:39:50.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:39:50.199 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:50.198+0000 7f94b340c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:50.200 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:50.199+0000 7f94b340c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:50.201 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:50.200+0000 7f94b340c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:50.331 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:50.330+0000 7f94b340c780 -1 Falling back to public interface 2026-02-17T18:39:50.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T18:39:50.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:39:50.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T18:39:50.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:39:50.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:39:50.412 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:39:50.412 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:39:50.412 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:39:50.412 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:39:50.413 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:39:50.629 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:50.628+0000 7f94b340c780 -1 osd.1 0 log_to_monitors true 2026-02-17T18:39:50.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:39:51.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:39:51.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:39:51.647 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:39:51.647 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:39:51.648 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:39:51.648 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:39:51.878 INFO:tasks.workunit.client.0.trial068.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1432632329,v1:127.0.0.1:6811/1432632329] [v2:127.0.0.1:6812/1432632329,v1:127.0.0.1:6813/1432632329] exists,up b862fb4c-fd2a-41b3-9035-b2552cd9189f 2026-02-17T18:39:51.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:119: TEST_network_ping_test2: run_osd td/network-ping 2 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:39:51.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:39:51.884 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:39:51.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:39:51.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-17T18:39:51.886 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:39:51.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4fe16765-0227-4692-a48a-79e5a94da7e7 2026-02-17T18:39:51.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4fe16765-0227-4692-a48a-79e5a94da7e7' 2026-02-17T18:39:51.886 INFO:tasks.workunit.client.0.trial068.stdout:add osd2 4fe16765-0227-4692-a48a-79e5a94da7e7 2026-02-17T18:39:51.886 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:39:51.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD3tZRpLUAmNRAAUrS6cUYf7zaVApSsodwiBw== 2026-02-17T18:39:51.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD3tZRpLUAmNRAAUrS6cUYf7zaVApSsodwiBw=="}' 2026-02-17T18:39:51.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4fe16765-0227-4692-a48a-79e5a94da7e7 -i td/network-ping/2/new.json 2026-02-17T18:39:52.123 INFO:tasks.workunit.client.0.trial068.stdout:2 2026-02-17T18:39:52.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-17T18:39:52.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3tZRpLUAmNRAAUrS6cUYf7zaVApSsodwiBw== --osd-uuid 4fe16765-0227-4692-a48a-79e5a94da7e7 2026-02-17T18:39:52.146 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.145+0000 7f5f4201a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:52.148 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.147+0000 7f5f4201a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:52.149 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.148+0000 7f5f4201a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:52.149 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.148+0000 7f5f4201a780 -1 bdev(0x2c16c00 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-17T18:39:52.149 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.148+0000 7f5f4201a780 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-17T18:39:52.459 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-17T18:39:52.459 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:39:52.460 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T18:39:52.460 INFO:tasks.workunit.client.0.trial068.stdout:adding osd2 key to auth repository 2026-02-17T18:39:52.460 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:39:52.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T18:39:52.801 INFO:tasks.workunit.client.0.trial068.stdout:start osd.2 2026-02-17T18:39:52.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=57ef7da7-61cb-436c-a097-ac5776955964 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.28424/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:39:52.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:39:52.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:39:52.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:39:52.816 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.815+0000 7f9bd1c2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:52.817 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.816+0000 7f9bd1c2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:52.818 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:52.817+0000 7f9bd1c2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:39:53.012 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:53.010+0000 7f9bd1c2f780 -1 Falling back to public interface 2026-02-17T18:39:53.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T18:39:53.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:39:53.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T18:39:53.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:39:53.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:39:53.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:39:53.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:39:53.042 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:39:53.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:39:53.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:39:53.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:39:53.321 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:39:53.319+0000 7f9bd1c2f780 -1 osd.2 0 log_to_monitors true 2026-02-17T18:39:54.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:39:54.265 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:39:54.265 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:39:54.266 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:39:54.266 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:39:54.266 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:39:54.488 INFO:tasks.workunit.client.0.trial068.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1647746450,v1:127.0.0.1:6819/1647746450] [v2:127.0.0.1:6820/1647746450,v1:127.0.0.1:6821/1647746450] exists,up 4fe16765-0227-4692-a48a-79e5a94da7e7 2026-02-17T18:39:54.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:39:54.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:39:54.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:39:54.490 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:121: TEST_network_ping_test2: sleep 5 2026-02-17T18:39:59.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:122: TEST_network_ping_test2: ceph osd crush add-bucket dc1 datacenter 2026-02-17T18:39:59.759 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'dc1' already exists 2026-02-17T18:39:59.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:123: TEST_network_ping_test2: ceph osd crush add-bucket dc2 datacenter 2026-02-17T18:40:00.039 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'dc2' already exists 2026-02-17T18:40:00.047 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:124: TEST_network_ping_test2: ceph osd crush add-bucket dc3 datacenter 2026-02-17T18:40:00.317 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'dc3' already exists 2026-02-17T18:40:00.324 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:125: TEST_network_ping_test2: ceph osd crush add-bucket rack1 rack 2026-02-17T18:40:00.598 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'rack1' already exists 2026-02-17T18:40:00.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:126: TEST_network_ping_test2: ceph osd crush add-bucket rack2 rack 2026-02-17T18:40:00.877 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'rack2' already exists 2026-02-17T18:40:00.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:127: TEST_network_ping_test2: ceph osd crush add-bucket rack3 rack 2026-02-17T18:40:01.156 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'rack3' already exists 2026-02-17T18:40:01.163 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:128: TEST_network_ping_test2: ceph osd crush add-bucket host1 host 2026-02-17T18:40:01.434 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'host1' already exists 2026-02-17T18:40:01.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:129: TEST_network_ping_test2: ceph osd crush add-bucket host2 host 2026-02-17T18:40:01.713 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'host2' already exists 2026-02-17T18:40:01.720 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:130: TEST_network_ping_test2: ceph osd crush add-bucket host3 host 2026-02-17T18:40:01.993 INFO:tasks.workunit.client.0.trial068.stderr:bucket 'host3' already exists 2026-02-17T18:40:02.000 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:131: TEST_network_ping_test2: ceph osd crush move dc1 root=default 2026-02-17T18:40:02.277 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -5 name 'dc1' to location {root=default} in crush map 2026-02-17T18:40:02.284 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:132: TEST_network_ping_test2: ceph osd crush move dc2 root=default 2026-02-17T18:40:02.560 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -6 name 'dc2' to location {root=default} in crush map 2026-02-17T18:40:02.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:133: TEST_network_ping_test2: ceph osd crush move dc3 root=default 2026-02-17T18:40:02.842 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -7 name 'dc3' to location {root=default} in crush map 2026-02-17T18:40:02.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:134: TEST_network_ping_test2: ceph osd crush move rack1 datacenter=dc1 2026-02-17T18:40:03.122 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -8 name 'rack1' to location {datacenter=dc1} in crush map 2026-02-17T18:40:03.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:135: TEST_network_ping_test2: ceph osd crush move rack2 datacenter=dc2 2026-02-17T18:40:03.401 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -9 name 'rack2' to location {datacenter=dc2} in crush map 2026-02-17T18:40:03.410 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:136: TEST_network_ping_test2: ceph osd crush move rack3 datacenter=dc3 2026-02-17T18:40:03.709 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -10 name 'rack3' to location {datacenter=dc3} in crush map 2026-02-17T18:40:03.717 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:137: TEST_network_ping_test2: ceph osd crush move host1 rack=rack1 2026-02-17T18:40:03.989 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -11 name 'host1' to location {rack=rack1} in crush map 2026-02-17T18:40:03.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:138: TEST_network_ping_test2: ceph osd crush move host2 rack=rack2 2026-02-17T18:40:04.275 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -12 name 'host2' to location {rack=rack2} in crush map 2026-02-17T18:40:04.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:139: TEST_network_ping_test2: ceph osd crush move host3 rack=rack3 2026-02-17T18:40:04.556 INFO:tasks.workunit.client.0.trial068.stderr:no need to move item id -13 name 'host3' to location {rack=rack3} in crush map 2026-02-17T18:40:04.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:140: TEST_network_ping_test2: ceph osd crush set osd.0 1.0 host=host1 2026-02-17T18:40:04.836 INFO:tasks.workunit.client.0.trial068.stderr:set item id 0 name 'osd.0' weight 1 at location {host=host1}: no change 2026-02-17T18:40:04.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:141: TEST_network_ping_test2: ceph osd crush set osd.1 1.0 host=host2 2026-02-17T18:40:05.117 INFO:tasks.workunit.client.0.trial068.stderr:set item id 1 name 'osd.1' weight 1 at location {host=host2}: no change 2026-02-17T18:40:05.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:142: TEST_network_ping_test2: ceph osd crush set osd.2 1.0 host=host3 2026-02-17T18:40:05.399 INFO:tasks.workunit.client.0.trial068.stderr:set item id 2 name 'osd.2' weight 1 at location {host=host3}: no change 2026-02-17T18:40:05.407 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:143: TEST_network_ping_test2: ceph osd crush rule create-simple myrule default host firstn 2026-02-17T18:40:05.681 INFO:tasks.workunit.client.0.trial068.stderr:rule myrule already exists 2026-02-17T18:40:05.689 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:145: TEST_network_ping_test2: create_pool foo 16 16 replicated myrule 2026-02-17T18:40:05.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 16 replicated myrule 2026-02-17T18:40:05.970 INFO:tasks.workunit.client.0.trial068.stderr:pool 'foo' already exists 2026-02-17T18:40:05.978 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:40:06.980 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:148: TEST_network_ping_test2: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-17T18:40:07.000 INFO:tasks.workunit.client.0.trial068.stdout:hints = 1 2026-02-17T18:40:07.000 INFO:tasks.workunit.client.0.trial068.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-17T18:40:07.000 INFO:tasks.workunit.client.0.trial068.stdout:Object prefix: benchmark_data_trial068_38809 2026-02-17T18:40:07.000 INFO:tasks.workunit.client.0.trial068.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T18:40:07.000 INFO:tasks.workunit.client.0.trial068.stdout: 0 0 0 0 0 0 - 0 2026-02-17T18:40:08.000 INFO:tasks.workunit.client.0.trial068.stdout: 1 16 11806 11790 46.0509 46.0547 0.00102047 0.00135486 2026-02-17T18:40:09.000 INFO:tasks.workunit.client.0.trial068.stdout: 2 16 23918 23902 46.68 47.3125 0.00093173 0.00133641 2026-02-17T18:40:10.000 INFO:tasks.workunit.client.0.trial068.stdout: 3 16 36613 36597 47.6487 49.5898 0.000819199 0.00131018 2026-02-17T18:40:11.000 INFO:tasks.workunit.client.0.trial068.stdout: 4 16 49242 49226 48.0686 49.332 0.00196346 0.00129878 2026-02-17T18:40:12.000 INFO:tasks.workunit.client.0.trial068.stdout: 5 16 62121 62105 48.5158 50.3086 0.00167719 0.00128685 2026-02-17T18:40:13.000 INFO:tasks.workunit.client.0.trial068.stdout: 6 16 74414 74398 48.4328 48.0195 0.0032086 0.00128907 2026-02-17T18:40:14.000 INFO:tasks.workunit.client.0.trial068.stdout: 7 16 86153 86137 48.0641 45.8555 0.00152844 0.00129824 2026-02-17T18:40:15.000 INFO:tasks.workunit.client.0.trial068.stdout: 8 15 96772 96757 47.2412 41.4844 0.00132725 0.00132161 2026-02-17T18:40:16.000 INFO:tasks.workunit.client.0.trial068.stdout: 9 16 107604 107588 46.6928 42.3086 0.00175979 0.00133714 2026-02-17T18:40:17.000 INFO:tasks.workunit.client.0.trial068.stdout: 10 6 118311 118305 46.2095 41.8633 0.00133547 0.00135119 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Total time run: 10.0009 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Total writes made: 118311 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Write size: 4096 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Object size: 4096 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Bandwidth (MB/sec): 46.211 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Stddev Bandwidth: 3.31755 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Max bandwidth (MB/sec): 50.3086 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Min bandwidth (MB/sec): 41.4844 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Average IOPS: 11830 2026-02-17T18:40:18.001 INFO:tasks.workunit.client.0.trial068.stdout:Stddev IOPS: 849.292 2026-02-17T18:40:18.002 INFO:tasks.workunit.client.0.trial068.stdout:Max IOPS: 12879 2026-02-17T18:40:18.002 INFO:tasks.workunit.client.0.trial068.stdout:Min IOPS: 10620 2026-02-17T18:40:18.002 INFO:tasks.workunit.client.0.trial068.stdout:Average Latency(s): 0.00135118 2026-02-17T18:40:18.002 INFO:tasks.workunit.client.0.trial068.stdout:Stddev Latency(s): 0.0006315 2026-02-17T18:40:18.002 INFO:tasks.workunit.client.0.trial068.stdout:Max latency(s): 0.0109071 2026-02-17T18:40:18.002 INFO:tasks.workunit.client.0.trial068.stdout:Min latency(s): 0.000426572 2026-02-17T18:40:18.005 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:151: TEST_network_ping_test2: sleep 10 2026-02-17T18:40:28.006 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:152: TEST_network_ping_test2: flush_pg_stats 2026-02-17T18:40:28.007 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:40:28.007 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:40:28.232 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:40:28.232 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:40:28.232 INFO:tasks.workunit.client.0.trial068.stderr:2' 2026-02-17T18:40:28.232 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:40:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:40:28.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:40:28.296 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-17T18:40:28.296 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-17T18:40:28.296 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-17T18:40:28.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:40:28.297 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:40:28.361 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672971 2026-02-17T18:40:28.362 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672971 2026-02-17T18:40:28.362 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672971' 2026-02-17T18:40:28.362 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:40:28.362 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:40:28.425 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509449 2026-02-17T18:40:28.425 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509449 2026-02-17T18:40:28.425 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672971 2-64424509449' 2026-02-17T18:40:28.425 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:40:28.425 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-17T18:40:28.426 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:40:28.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:40:28.427 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-17T18:40:28.427 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:40:28.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-17T18:40:28.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-17T18:40:28.428 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836490 2026-02-17T18:40:28.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:40:28.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-17T18:40:28.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:40:29.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:40:29.656 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:40:29.878 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-17T18:40:29.879 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:40:29.879 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672971 2026-02-17T18:40:29.879 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:40:29.880 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:40:29.880 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672971 2026-02-17T18:40:29.880 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:40:29.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672971 2026-02-17T18:40:29.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-17T18:40:29.882 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 42949672971 2026-02-17T18:40:29.882 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:40:30.110 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-17T18:40:30.110 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:40:30.111 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-64424509449 2026-02-17T18:40:30.111 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:40:30.111 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:40:30.112 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509449 2026-02-17T18:40:30.112 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:40:30.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509449 2026-02-17T18:40:30.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-17T18:40:30.113 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 64424509449 2026-02-17T18:40:30.114 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:40:30.334 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-17T18:40:30.335 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: ceph health 2026-02-17T18:40:30.335 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: tee td/network-ping/health 2026-02-17T18:40:30.616 INFO:tasks.workunit.client.0.trial068.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.242ms); Slow OSD heartbeats on front (longest 0.262ms) 2026-02-17T18:40:30.623 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:155: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats' td/network-ping/health 2026-02-17T18:40:30.626 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: ceph health detail 2026-02-17T18:40:30.626 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: tee td/network-ping/health 2026-02-17T18:40:30.928 INFO:tasks.workunit.client.0.trial068.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.242ms); Slow OSD heartbeats on front (longest 0.262ms) 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout:[WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 0.242ms) 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.242 msec 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.209 msec 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.203 msec 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.180 msec possibly improving 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.151 msec 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.109 msec possibly improving 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout:[WRN] OSD_SLOW_PING_TIME_FRONT: Slow OSD heartbeats on front (longest 0.262ms) 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.262 msec possibly improving 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.223 msec 2026-02-17T18:40:30.929 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.212 msec 2026-02-17T18:40:30.930 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.205 msec 2026-02-17T18:40:30.930 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.151 msec 2026-02-17T18:40:30.930 INFO:tasks.workunit.client.0.trial068.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.132 msec 2026-02-17T18:40:30.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:158: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_BACK td/network-ping/health 2026-02-17T18:40:30.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:159: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_FRONT td/network-ping/health 2026-02-17T18:40:30.937 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:160: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats on front from osd[.][0-2] [[]dc[1-3],rack[1-3][]] to osd[.][0-2] [[]dc[1-3],rack[1-3][]]' td/network-ping/health 2026-02-17T18:40:30.938 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:162: TEST_network_ping_test2: rm -f td/network-ping/health 2026-02-17T18:40:30.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-17T18:40:30.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T18:40:30.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:40:30.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T18:40:30.940 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:40:30.940 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:40:30.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:40:30.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:40:30.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:40:31.047 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:40:31.047 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:40:31.048 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:40:31.048 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:40:31.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:40:31.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:40:31.050 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:40:31.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:31.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:40:31.051 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:40:31.051 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:31.051 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:40:31.052 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:40:31.052 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T18:40:31.703 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:40:31.703 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:31.703 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:40:31.703 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.28424 2026-02-17T18:40:31.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:40:31.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:40:31.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T18:40:31.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/network-ping 0 2026-02-17T18:40:31.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-17T18:40:31.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T18:40:31.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-17T18:40:31.706 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:40:31.706 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:40:31.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:40:31.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:40:31.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:40:31.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:40:31.707 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:40:31.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:40:31.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:40:31.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:40:31.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:40:31.709 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:40:31.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:31.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:40:31.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:40:31.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:31.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:40:31.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T18:40:31.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-17T18:40:31.713 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:40:31.713 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:31.713 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.28424 2026-02-17T18:40:31.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.28424 2026-02-17T18:40:31.714 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:40:31.714 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:40:31.714 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T18:40:31.715 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T18:40:31.715 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T18:40:31.781 INFO:tasks.workunit:Running workunit misc/ok-to-stop.sh... 2026-02-17T18:40:31.781 DEBUG:teuthology.orchestra.run.trial068:workunit test misc/ok-to-stop.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh 2026-02-17T18:40:31.845 INFO:tasks.workunit.client.0.trial068.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T18:40:31.850 INFO:tasks.workunit.client.0.trial068.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T18:40:31.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/ok-to-stop 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:6: run: local dir=td/ok-to-stop 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:7: run: shift 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: export CEPH_MON_A=127.0.0.1:7150 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: CEPH_MON_A=127.0.0.1:7150 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: export CEPH_MON_B=127.0.0.1:7151 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: CEPH_MON_B=127.0.0.1:7151 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: export CEPH_MON_C=127.0.0.1:7152 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: CEPH_MON_C=127.0.0.1:7152 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: export CEPH_MON_D=127.0.0.1:7153 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: CEPH_MON_D=127.0.0.1:7153 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: export CEPH_MON_E=127.0.0.1:7154 2026-02-17T18:40:31.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: CEPH_MON_E=127.0.0.1:7154 2026-02-17T18:40:31.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:14: run: export CEPH_ARGS 2026-02-17T18:40:31.853 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: uuidgen 2026-02-17T18:40:31.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: CEPH_ARGS+='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none ' 2026-02-17T18:40:31.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: export 'ORIG_CEPH_ARGS=--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none ' 2026-02-17T18:40:31.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: ORIG_CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none ' 2026-02-17T18:40:31.853 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: set 2026-02-17T18:40:31.853 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T18:40:31.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: local 'funcs=TEST_0_mds 2026-02-17T18:40:31.856 INFO:tasks.workunit.client.0.trial068.stderr:TEST_0_osd 2026-02-17T18:40:31.856 INFO:tasks.workunit.client.0.trial068.stderr:TEST_1_mon_checks 2026-02-17T18:40:31.856 INFO:tasks.workunit.client.0.trial068.stderr:TEST_2_mons_checks 2026-02-17T18:40:31.856 INFO:tasks.workunit.client.0.trial068.stderr:TEST_3_mons_checks 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:TEST_4_mons_checks 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:TEST_5_mons_checks' 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:40:31.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:40:31.858 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:31.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:31.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:40:31.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:40:31.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:40:31.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T18:40:31.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:40:31.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:31.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:31.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39196 2026-02-17T18:40:31.863 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:40:31.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T18:40:31.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_mds td/ok-to-stop 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:208: TEST_0_mds: local dir=td/ok-to-stop 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:210: TEST_0_mds: CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:212: TEST_0_mds: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T18:40:31.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T18:40:31.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:40:31.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:31.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:31.883 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:31.883 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:31.883 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:31.884 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:31.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T18:40:31.907 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:40:31.909 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:40:31.909 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:40:31.909 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:40:31.909 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:40:31.909 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:40:31.910 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:40:31.910 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:40:31.910 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:40:31.910 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:40:31.910 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:31.910 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:31.910 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:40:31.911 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:40:31.911 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get fsid 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:40:31.953 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:40:31.954 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:40:31.955 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:31.955 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:31.955 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:40:31.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:40:31.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get mon_host 2026-02-17T18:40:31.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:213: TEST_0_mds: run_mgr td/ok-to-stop x 2026-02-17T18:40:31.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-17T18:40:31.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:40:31.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:40:31.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:40:31.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-17T18:40:31.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:40:32.094 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:40:32.094 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:32.094 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:32.095 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:32.095 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:32.095 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:32.095 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:32.095 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:40:32.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:40:32.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:214: TEST_0_mds: run_osd td/ok-to-stop 0 2026-02-17T18:40:32.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T18:40:32.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:40:32.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:40:32.114 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:40:32.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=392d9f8c-7b00-4925-b491-fd2aa940cc02 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 392d9f8c-7b00-4925-b491-fd2aa940cc02' 2026-02-17T18:40:32.116 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 392d9f8c-7b00-4925-b491-fd2aa940cc02 2026-02-17T18:40:32.117 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:40:32.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAgtpRpVJovBxAAoDHC2XIdj2Gd5N5fJ7RNVw== 2026-02-17T18:40:32.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAgtpRpVJovBxAAoDHC2XIdj2Gd5N5fJ7RNVw=="}' 2026-02-17T18:40:32.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 392d9f8c-7b00-4925-b491-fd2aa940cc02 -i td/ok-to-stop/0/new.json 2026-02-17T18:40:32.222 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:40:32.228 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-17T18:40:32.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgtpRpVJovBxAAoDHC2XIdj2Gd5N5fJ7RNVw== --osd-uuid 392d9f8c-7b00-4925-b491-fd2aa940cc02 2026-02-17T18:40:32.245 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.244+0000 7fee4ba33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:32.246 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.245+0000 7fee4ba33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:32.247 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.246+0000 7fee4ba33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:32.248 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.247+0000 7fee4ba33780 -1 bdev(0x2e72c00 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:40:32.248 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.247+0000 7fee4ba33780 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-17T18:40:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-17T18:40:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:40:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:40:32.655 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:40:32.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:40:32.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:40:32.759 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:40:32.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:40:32.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:40:32.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:40:32.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:40:32.771 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.770+0000 7fa78ba0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:32.773 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.772+0000 7fa78ba0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:32.774 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.773+0000 7fa78ba0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:32.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:40:32.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:40:32.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:40:32.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:40:32.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:40:32.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:32.858 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:40:32.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:40:32.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:32.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:40:32.954 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:40:32.988 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:32.987+0000 7fa78ba0e780 -1 Falling back to public interface 2026-02-17T18:40:33.395 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:33.394+0000 7fa78ba0e780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:40:33.955 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:40:33.955 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:33.956 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:40:33.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:40:33.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:33.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:40:34.051 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3276758822,v1:127.0.0.1:6801/3276758822] [v2:127.0.0.1:6802/3276758822,v1:127.0.0.1:6803/3276758822] exists,up 392d9f8c-7b00-4925-b491-fd2aa940cc02 2026-02-17T18:40:34.053 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:40:34.053 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:40:34.053 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:215: TEST_0_mds: run_mds td/ok-to-stop a 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=a 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/a 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:34.054 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:34.055 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:34.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id a --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-17T18:40:34.067 INFO:tasks.workunit.client.0.trial068.stdout:starting mds.a at 2026-02-17T18:40:34.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:217: TEST_0_mds: ceph osd pool create meta 1 2026-02-17T18:40:34.224 INFO:tasks.workunit.client.0.trial068.stderr:pool 'meta' already exists 2026-02-17T18:40:34.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:218: TEST_0_mds: ceph osd pool create data 1 2026-02-17T18:40:34.472 INFO:tasks.workunit.client.0.trial068.stderr:pool 'data' already exists 2026-02-17T18:40:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:219: TEST_0_mds: ceph fs new myfs meta data 2026-02-17T18:40:34.818 INFO:tasks.workunit.client.0.trial068.stderr:filesystem 'myfs' already exists 2026-02-17T18:40:34.826 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:220: TEST_0_mds: sleep 5 2026-02-17T18:40:39.827 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:222: TEST_0_mds: ceph mds ok-to-stop a 2026-02-17T18:40:40.035 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 2026-02-17T18:40:40.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:223: TEST_0_mds: ceph mds ok-to-stop a dne 2026-02-17T18:40:40.238 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 2026-02-17T18:40:40.241 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:224: TEST_0_mds: ceph mds ok-to-stop dne 2026-02-17T18:40:40.554 INFO:tasks.workunit.client.0.trial068.stderr:should be safe to stop [dne] 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:226: TEST_0_mds: run_mds td/ok-to-stop b 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=b 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/b 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:40.563 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:40.564 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:40.564 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:40.564 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:40.564 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:40.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id b --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-17T18:40:40.576 INFO:tasks.workunit.client.0.trial068.stdout:starting mds.b at 2026-02-17T18:40:40.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:227: TEST_0_mds: sleep 5 2026-02-17T18:40:45.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:229: TEST_0_mds: ceph mds ok-to-stop a 2026-02-17T18:40:45.893 INFO:tasks.workunit.client.0.trial068.stderr:should be safe to stop [a] 2026-02-17T18:40:45.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:230: TEST_0_mds: ceph mds ok-to-stop b 2026-02-17T18:40:46.212 INFO:tasks.workunit.client.0.trial068.stderr:should be safe to stop [b] 2026-02-17T18:40:46.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:231: TEST_0_mds: ceph mds ok-to-stop a b 2026-02-17T18:40:46.421 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 4135 2026-02-17T18:40:46.424 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:232: TEST_0_mds: ceph mds ok-to-stop a dne1 dne2 2026-02-17T18:40:46.730 INFO:tasks.workunit.client.0.trial068.stderr:should be safe to stop [a,dne1,dne2] 2026-02-17T18:40:46.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:233: TEST_0_mds: ceph mds ok-to-stop b dne 2026-02-17T18:40:47.043 INFO:tasks.workunit.client.0.trial068.stderr:should be safe to stop [b,dne] 2026-02-17T18:40:47.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:234: TEST_0_mds: ceph mds ok-to-stop a b dne 2026-02-17T18:40:47.260 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 4135 2026-02-17T18:40:47.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:235: TEST_0_mds: ceph mds ok-to-stop dne1 dne2 2026-02-17T18:40:47.569 INFO:tasks.workunit.client.0.trial068.stderr:should be safe to stop [dne1,dne2] 2026-02-17T18:40:47.577 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:237: TEST_0_mds: kill_daemons td/ok-to-stop KILL mds.a 2026-02-17T18:40:47.578 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:40:47.578 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:40:47.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:40:47.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:40:47.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:40:47.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:40:47.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T18:40:47.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:40:47.685 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:40:47.685 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:40:47.685 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:40:47.685 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:40:47.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:40:47.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T18:40:47.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:40:47.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:40:47.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:40:47.793 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:40:47.794 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:40:47.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:40:47.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:40:47.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:40:47.799 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:40:47.799 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:40:47.800 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:40:47.800 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:40:47.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:40:47.801 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:40:47.801 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:40:47.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:47.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:40:47.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:40:47.803 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:47.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:40:47.804 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:40:47.804 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:40:47.808 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:40:47.809 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:47.809 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:47.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:40:47.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:40:47.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:40:47.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:40:47.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:40:47.812 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:40:47.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:40:47.813 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:40:47.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:40:47.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:40:47.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:40:47.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:47.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:40:47.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:40:47.816 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:40:47.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:40:47.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:40:47.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:40:47.818 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:40:47.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:47.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:47.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:40:47.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:40:47.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:40:47.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T18:40:47.821 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:40:47.821 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:47.821 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:47.821 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39196 2026-02-17T18:40:47.822 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:40:47.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:40:47.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:40:47.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_osd td/ok-to-stop 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:241: TEST_0_osd: local dir=td/ok-to-stop 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:243: TEST_0_osd: CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:245: TEST_0_osd: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T18:40:47.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T18:40:47.840 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:40:47.840 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:47.840 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:47.840 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:47.840 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:47.840 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:47.840 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:47.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T18:40:47.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:40:47.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:40:47.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:40:47.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:40:47.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:40:47.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:40:47.864 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:40:47.864 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:40:47.865 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:40:47.865 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:40:47.865 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:47.865 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:47.865 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:40:47.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:40:47.866 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get fsid 2026-02-17T18:40:47.910 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:40:47.910 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:40:47.911 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:40:47.911 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:40:47.911 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:40:47.912 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:40:47.912 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:40:47.912 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:40:47.912 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:40:47.912 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:47.912 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:47.913 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:40:47.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:40:47.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get mon_host 2026-02-17T18:40:47.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:246: TEST_0_osd: run_mgr td/ok-to-stop x 2026-02-17T18:40:47.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-17T18:40:47.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:40:47.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:40:47.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:40:47.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-17T18:40:47.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:40:48.052 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:40:48.053 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:48.053 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:48.053 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:48.053 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:48.053 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:48.053 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:48.054 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:40:48.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:247: TEST_0_osd: run_osd td/ok-to-stop 0 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:40:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-17T18:40:48.074 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:40:48.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dc523fdd-e80d-4af0-9542-ceb00f5d8eb2 2026-02-17T18:40:48.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dc523fdd-e80d-4af0-9542-ceb00f5d8eb2' 2026-02-17T18:40:48.075 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 dc523fdd-e80d-4af0-9542-ceb00f5d8eb2 2026-02-17T18:40:48.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:40:48.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAwtpRpK+jEBBAAmMvERrcS0/xs2NmTGp6R6w== 2026-02-17T18:40:48.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAwtpRpK+jEBBAAmMvERrcS0/xs2NmTGp6R6w=="}' 2026-02-17T18:40:48.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dc523fdd-e80d-4af0-9542-ceb00f5d8eb2 -i td/ok-to-stop/0/new.json 2026-02-17T18:40:48.181 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:40:48.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-17T18:40:48.188 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAwtpRpK+jEBBAAmMvERrcS0/xs2NmTGp6R6w== --osd-uuid dc523fdd-e80d-4af0-9542-ceb00f5d8eb2 2026-02-17T18:40:48.202 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.201+0000 7feab7dd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:48.204 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.203+0000 7feab7dd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:48.205 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.204+0000 7feab7dd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:48.205 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.204+0000 7feab7dd8780 -1 bdev(0x4070c00 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:40:48.205 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.204+0000 7feab7dd8780 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-17T18:40:48.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-17T18:40:48.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:40:48.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:40:48.672 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:40:48.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:40:48.775 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:40:48.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:40:48.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:40:48.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:40:48.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:40:48.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:40:48.789 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.788+0000 7f89bdc3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:48.790 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.789+0000 7f89bdc3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:48.791 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.790+0000 7f89bdc3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:48.869 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:40:48.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:40:48.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:40:48.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:40:48.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:40:48.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:48.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:40:48.870 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:40:48.871 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:48.871 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:40:48.965 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:40:48.993 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:48.992+0000 7f89bdc3d780 -1 Falling back to public interface 2026-02-17T18:40:49.401 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:49.400+0000 7f89bdc3d780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:40:49.966 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:40:49.966 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:49.966 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:40:49.967 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:40:49.967 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:49.967 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:40:50.064 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1073349119,v1:127.0.0.1:6801/1073349119] [v2:127.0.0.1:6802/1073349119,v1:127.0.0.1:6803/1073349119] exists,up dc523fdd-e80d-4af0-9542-ceb00f5d8eb2 2026-02-17T18:40:50.068 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:40:50.068 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:248: TEST_0_osd: run_osd td/ok-to-stop 1 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/1 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:40:50.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/1' 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/1/journal' 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:50.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/1 2026-02-17T18:40:50.071 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:40:50.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3370a6f8-3b40-4263-86ab-2f695da98426 2026-02-17T18:40:50.072 INFO:tasks.workunit.client.0.trial068.stdout:add osd1 3370a6f8-3b40-4263-86ab-2f695da98426 2026-02-17T18:40:50.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 3370a6f8-3b40-4263-86ab-2f695da98426' 2026-02-17T18:40:50.072 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:40:50.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAytpRp/peKBBAAIgH+UPsWqnEZJn7njhJpTg== 2026-02-17T18:40:50.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAytpRp/peKBBAAIgH+UPsWqnEZJn7njhJpTg=="}' 2026-02-17T18:40:50.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3370a6f8-3b40-4263-86ab-2f695da98426 -i td/ok-to-stop/1/new.json 2026-02-17T18:40:50.177 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:40:50.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/1/new.json 2026-02-17T18:40:50.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAytpRp/peKBBAAIgH+UPsWqnEZJn7njhJpTg== --osd-uuid 3370a6f8-3b40-4263-86ab-2f695da98426 2026-02-17T18:40:50.199 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.198+0000 7f581044a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:50.200 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.199+0000 7f581044a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:50.201 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.200+0000 7f581044a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:50.202 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.201+0000 7f581044a780 -1 bdev(0x3526c00 td/ok-to-stop/1/block) open stat got: (1) Operation not permitted 2026-02-17T18:40:50.202 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.201+0000 7f581044a780 -1 bluestore(td/ok-to-stop/1) _read_fsid unparsable uuid 2026-02-17T18:40:50.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/1/keyring 2026-02-17T18:40:50.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:40:50.520 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T18:40:50.521 INFO:tasks.workunit.client.0.trial068.stdout:adding osd1 key to auth repository 2026-02-17T18:40:50.521 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:40:50.835 INFO:tasks.workunit.client.0.trial068.stdout:start osd.1 2026-02-17T18:40:50.836 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T18:40:50.836 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:40:50.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:40:50.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:40:50.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:40:50.851 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.849+0000 7fda8ec37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:50.852 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.851+0000 7fda8ec37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:50.853 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.852+0000 7fda8ec37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:50.996 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:50.994+0000 7fda8ec37780 -1 Falling back to public interface 2026-02-17T18:40:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T18:40:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:40:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T18:40:51.061 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:40:51.061 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:40:51.061 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:51.061 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:40:51.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:40:51.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:51.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:40:51.280 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:40:51.376 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:51.375+0000 7fda8ec37780 -1 osd.1 0 log_to_monitors true 2026-02-17T18:40:52.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:40:52.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:52.282 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:40:52.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:40:52.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:52.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:40:52.512 INFO:tasks.workunit.client.0.trial068.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4109810961,v1:127.0.0.1:6811/4109810961] [v2:127.0.0.1:6812/4109810961,v1:127.0.0.1:6813/4109810961] exists,up 3370a6f8-3b40-4263-86ab-2f695da98426 2026-02-17T18:40:52.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:40:52.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:40:52.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:40:52.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:249: TEST_0_osd: run_osd td/ok-to-stop 2 2026-02-17T18:40:52.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T18:40:52.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/2 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/2' 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/2/journal' 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:40:52.517 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T18:40:52.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/2 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:40:52.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9362b2c9-6e3a-4b26-a4ef-73768374531a 2026-02-17T18:40:52.520 INFO:tasks.workunit.client.0.trial068.stdout:add osd2 9362b2c9-6e3a-4b26-a4ef-73768374531a 2026-02-17T18:40:52.520 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 9362b2c9-6e3a-4b26-a4ef-73768374531a' 2026-02-17T18:40:52.520 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:40:52.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA0tpRpjJo8HxAAAS3hQtXL24ExRlOoP9EpcA== 2026-02-17T18:40:52.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA0tpRpjJo8HxAAAS3hQtXL24ExRlOoP9EpcA=="}' 2026-02-17T18:40:52.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9362b2c9-6e3a-4b26-a4ef-73768374531a -i td/ok-to-stop/2/new.json 2026-02-17T18:40:52.752 INFO:tasks.workunit.client.0.trial068.stdout:2 2026-02-17T18:40:52.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/2/new.json 2026-02-17T18:40:52.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA0tpRpjJo8HxAAAS3hQtXL24ExRlOoP9EpcA== --osd-uuid 9362b2c9-6e3a-4b26-a4ef-73768374531a 2026-02-17T18:40:52.776 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:52.774+0000 7fd17fa0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:52.777 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:52.776+0000 7fd17fa0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:52.778 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:52.777+0000 7fd17fa0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:52.778 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:52.777+0000 7fd17fa0c780 -1 bdev(0x3ea4c00 td/ok-to-stop/2/block) open stat got: (1) Operation not permitted 2026-02-17T18:40:52.779 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:52.777+0000 7fd17fa0c780 -1 bluestore(td/ok-to-stop/2) _read_fsid unparsable uuid 2026-02-17T18:40:53.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/2/keyring 2026-02-17T18:40:53.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:40:53.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T18:40:53.121 INFO:tasks.workunit.client.0.trial068.stdout:adding osd2 key to auth repository 2026-02-17T18:40:53.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:40:53.435 INFO:tasks.workunit.client.0.trial068.stdout:start osd.2 2026-02-17T18:40:53.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T18:40:53.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:40:53.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:40:53.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:40:53.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:40:53.449 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:53.447+0000 7f05e67d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:53.450 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:53.449+0000 7f05e67d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:53.451 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:53.450+0000 7f05e67d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:53.580 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:53.579+0000 7f05e67d8780 -1 Falling back to public interface 2026-02-17T18:40:53.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T18:40:53.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:40:53.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T18:40:53.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:40:53.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:40:53.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:53.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:40:53.660 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:40:53.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:53.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:40:53.883 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:53.882+0000 7f05e67d8780 -1 osd.2 0 log_to_monitors true 2026-02-17T18:40:53.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:40:54.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:40:54.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:54.886 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:40:54.887 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:40:54.887 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:54.887 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:40:55.112 INFO:tasks.workunit.client.0.trial068.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/455671762,v1:127.0.0.1:6819/455671762] [v2:127.0.0.1:6820/455671762,v1:127.0.0.1:6821/455671762] exists,up 9362b2c9-6e3a-4b26-a4ef-73768374531a 2026-02-17T18:40:55.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:40:55.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:40:55.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:40:55.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:250: TEST_0_osd: run_osd td/ok-to-stop 3 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/3 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/3' 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/3/journal' 2026-02-17T18:40:55.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:40:55.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/3 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:40:55.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0bc47664-5a5c-439f-9617-16bceb3a5194 2026-02-17T18:40:55.120 INFO:tasks.workunit.client.0.trial068.stdout:add osd3 0bc47664-5a5c-439f-9617-16bceb3a5194 2026-02-17T18:40:55.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 0bc47664-5a5c-439f-9617-16bceb3a5194' 2026-02-17T18:40:55.120 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:40:55.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA3tpRpPLFbBxAA4m4J9IV2iKTFGuGfp3k4JA== 2026-02-17T18:40:55.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA3tpRpPLFbBxAA4m4J9IV2iKTFGuGfp3k4JA=="}' 2026-02-17T18:40:55.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0bc47664-5a5c-439f-9617-16bceb3a5194 -i td/ok-to-stop/3/new.json 2026-02-17T18:40:55.353 INFO:tasks.workunit.client.0.trial068.stdout:3 2026-02-17T18:40:55.360 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/3/new.json 2026-02-17T18:40:55.361 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA3tpRpPLFbBxAA4m4J9IV2iKTFGuGfp3k4JA== --osd-uuid 0bc47664-5a5c-439f-9617-16bceb3a5194 2026-02-17T18:40:55.376 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:55.374+0000 7ffb97c13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:55.377 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:55.376+0000 7ffb97c13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:55.379 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:55.377+0000 7ffb97c13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:55.379 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:55.377+0000 7ffb97c13780 -1 bdev(0x3666c00 td/ok-to-stop/3/block) open stat got: (1) Operation not permitted 2026-02-17T18:40:55.379 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:55.377+0000 7ffb97c13780 -1 bluestore(td/ok-to-stop/3) _read_fsid unparsable uuid 2026-02-17T18:40:55.717 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/3/keyring 2026-02-17T18:40:55.718 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:40:55.718 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-17T18:40:55.719 INFO:tasks.workunit.client.0.trial068.stdout:adding osd3 key to auth repository 2026-02-17T18:40:55.719 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:40:56.038 INFO:tasks.workunit.client.0.trial068.stdout:start osd.3 2026-02-17T18:40:56.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-17T18:40:56.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:40:56.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:40:56.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:40:56.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:40:56.053 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:56.052+0000 7f397ba59780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:56.055 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:56.053+0000 7f397ba59780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:56.055 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:56.054+0000 7f397ba59780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:40:56.177 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:56.175+0000 7f397ba59780 -1 Falling back to public interface 2026-02-17T18:40:56.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-17T18:40:56.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:40:56.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-17T18:40:56.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:40:56.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:40:56.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:56.263 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:40:56.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:40:56.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:56.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-17T18:40:56.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:40:56.518 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:40:56.517+0000 7f397ba59780 -1 osd.3 0 log_to_monitors true 2026-02-17T18:40:57.490 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:40:57.490 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:40:57.491 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:40:57.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:40:57.492 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:40:57.492 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-17T18:40:57.724 INFO:tasks.workunit.client.0.trial068.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/431820914,v1:127.0.0.1:6827/431820914] [v2:127.0.0.1:6828/431820914,v1:127.0.0.1:6829/431820914] exists,up 0bc47664-5a5c-439f-9617-16bceb3a5194 2026-02-17T18:40:57.724 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:40:57.725 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:40:57.725 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:40:57.725 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:252: TEST_0_osd: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-17T18:40:58.011 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:253: TEST_0_osd: ceph osd pool create ec erasure ec-profile 2026-02-17T18:40:58.408 INFO:tasks.workunit.client.0.trial068.stderr:pool 'ec' already exists 2026-02-17T18:40:58.418 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:255: TEST_0_osd: wait_for_clean 2026-02-17T18:40:58.418 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:40:58.419 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:40:58.419 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:40:58.419 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:40:58.420 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:40:58.420 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:40:58.420 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:40:58.420 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:40:58.420 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:40:58.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:40:58.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:40:58.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:40:58.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:40:58.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:40:58.506 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:40:58.736 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:40:58.736 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:40:58.736 INFO:tasks.workunit.client.0.trial068.stderr:2 2026-02-17T18:40:58.737 INFO:tasks.workunit.client.0.trial068.stderr:3' 2026-02-17T18:40:58.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:40:58.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:40:58.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:40:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-17T18:40:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-17T18:40:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T18:40:58.798 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:40:58.798 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:40:58.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-17T18:40:58.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-17T18:40:58.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668' 2026-02-17T18:40:58.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:40:58.861 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:40:58.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-17T18:40:58.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-17T18:40:58.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668 2-60129542147' 2026-02-17T18:40:58.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:40:58.924 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T18:40:58.986 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-17T18:40:58.986 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-17T18:40:58.986 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668 2-60129542147 3-77309411331' 2026-02-17T18:40:58.986 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:40:58.986 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-17T18:40:58.986 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:40:58.987 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:40:58.987 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-17T18:40:58.987 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:40:58.988 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-17T18:40:58.989 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836484 2026-02-17T18:40:58.989 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T18:40:58.989 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:40:59.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T18:40:59.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:41:00.216 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:41:00.216 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:41:00.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T18:41:00.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:00.442 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705668 2026-02-17T18:41:00.442 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:00.442 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:41:00.443 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-17T18:41:00.443 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:00.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705668 2026-02-17T18:41:00.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T18:41:00.444 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705668 2026-02-17T18:41:00.444 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:41:00.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T18:41:00.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:00.673 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542147 2026-02-17T18:41:00.673 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:00.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:41:00.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-17T18:41:00.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:00.676 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542147 2026-02-17T18:41:00.676 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T18:41:00.676 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 60129542147 2026-02-17T18:41:00.676 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:41:00.900 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-17T18:41:00.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:00.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411331 2026-02-17T18:41:00.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:00.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T18:41:00.902 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411331 2026-02-17T18:41:00.902 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:00.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-17T18:41:00.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411331' 2026-02-17T18:41:00.904 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.3 seq 77309411331 2026-02-17T18:41:00.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T18:41:01.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411331 -lt 77309411331 2026-02-17T18:41:01.131 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:41:01.131 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:01.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:01.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-17T18:41:01.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:41:01.445 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:41:01.445 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:41:01.446 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:41:01.446 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:41:01.446 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:41:01.446 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:41:01.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-17T18:41:01.664 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:41:01.664 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:01.664 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:01.978 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-17T18:41:01.978 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:41:01.978 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:41:01.978 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:258: TEST_0_osd: ceph osd pool set ec min_size 3 2026-02-17T18:41:02.320 INFO:tasks.workunit.client.0.trial068.stderr:set pool 1 min_size to 3 2026-02-17T18:41:02.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:259: TEST_0_osd: wait_for_clean 2026-02-17T18:41:02.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:41:02.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:41:02.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:41:02.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:41:02.332 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:41:02.332 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:41:02.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:41:02.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:41:02.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:41:02.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:41:02.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:41:02.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:41:02.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:41:02.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:41:02.416 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:41:02.639 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:41:02.639 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:41:02.639 INFO:tasks.workunit.client.0.trial068.stderr:2 2026-02-17T18:41:02.639 INFO:tasks.workunit.client.0.trial068.stderr:3' 2026-02-17T18:41:02.639 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:41:02.639 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:02.640 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:41:02.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-17T18:41:02.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-17T18:41:02.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T18:41:02.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:02.702 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:41:02.764 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-17T18:41:02.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-17T18:41:02.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-38654705671' 2026-02-17T18:41:02.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:02.765 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:41:02.827 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-17T18:41:02.827 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-17T18:41:02.827 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-38654705671 2-60129542150' 2026-02-17T18:41:02.827 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:02.827 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T18:41:02.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-17T18:41:02.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-17T18:41:02.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-38654705671 2-60129542150 3-77309411334' 2026-02-17T18:41:02.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:02.891 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836487 2026-02-17T18:41:02.891 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:02.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:41:02.892 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-17T18:41:02.892 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:02.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836487 2026-02-17T18:41:02.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T18:41:02.894 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836487 2026-02-17T18:41:02.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:41:03.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-17T18:41:03.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:41:04.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:41:04.120 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:41:04.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T18:41:04.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:04.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705671 2026-02-17T18:41:04.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:04.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:41:04.356 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-17T18:41:04.356 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:04.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705671 2026-02-17T18:41:04.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T18:41:04.358 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705671 2026-02-17T18:41:04.358 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:41:04.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T18:41:04.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:04.579 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542150 2026-02-17T18:41:04.579 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:04.579 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:41:04.580 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-17T18:41:04.580 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:04.581 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542150 2026-02-17T18:41:04.581 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 60129542150 2026-02-17T18:41:04.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-17T18:41:04.582 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:41:04.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-17T18:41:04.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:04.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411334 2026-02-17T18:41:04.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:04.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T18:41:04.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411334 2026-02-17T18:41:04.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:04.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-17T18:41:04.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411334' 2026-02-17T18:41:04.813 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.3 seq 77309411334 2026-02-17T18:41:04.813 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T18:41:05.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-17T18:41:05.036 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:41:05.037 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:05.037 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:05.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-17T18:41:05.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:41:05.349 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:41:05.349 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:41:05.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:41:05.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:41:05.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:41:05.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:41:05.570 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-17T18:41:05.571 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:41:05.571 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:05.571 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:05.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-17T18:41:05.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:41:05.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:41:05.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:261: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-17T18:41:06.101 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:06.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:262: TEST_0_osd: ceph osd ok-to-stop 1 2026-02-17T18:41:06.323 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:06.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:263: TEST_0_osd: ceph osd ok-to-stop 2 2026-02-17T18:41:06.549 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:06.556 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:264: TEST_0_osd: ceph osd ok-to-stop 3 2026-02-17T18:41:06.773 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:06.780 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:265: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-17T18:41:06.935 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":false,"osds":[0,1],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:06.936 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T18:41:06.938 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:266: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-17T18:41:07.094 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":false,"osds":[2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:07.095 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T18:41:07.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-17T18:41:07.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: grep '[0]' 2026-02-17T18:41:07.322 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:07.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: ceph osd ok-to-stop 1 --max 2 2026-02-17T18:41:07.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: grep '[1]' 2026-02-17T18:41:07.544 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:07.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:271: TEST_0_osd: ceph osd pool set ec min_size 2 2026-02-17T18:41:07.853 INFO:tasks.workunit.client.0.trial068.stderr:set pool 1 min_size to 2 2026-02-17T18:41:07.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:272: TEST_0_osd: wait_for_clean 2026-02-17T18:41:07.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:41:07.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:41:07.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:41:07.867 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:41:07.868 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:41:07.868 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:41:07.868 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:41:07.868 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:41:07.868 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:41:07.949 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:41:07.949 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:41:07.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:41:07.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:41:07.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:41:07.950 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:41:08.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:41:08.174 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:41:08.175 INFO:tasks.workunit.client.0.trial068.stderr:2 2026-02-17T18:41:08.175 INFO:tasks.workunit.client.0.trial068.stderr:3' 2026-02-17T18:41:08.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:41:08.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:08.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:41:08.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-17T18:41:08.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-17T18:41:08.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-17T18:41:08.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:08.237 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:41:08.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-17T18:41:08.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-17T18:41:08.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705674' 2026-02-17T18:41:08.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:08.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:41:08.365 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-17T18:41:08.365 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-17T18:41:08.365 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-60129542153' 2026-02-17T18:41:08.366 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:08.366 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T18:41:08.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-17T18:41:08.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-17T18:41:08.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-60129542153 3-77309411337' 2026-02-17T18:41:08.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:08.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-17T18:41:08.429 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:08.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:41:08.429 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-17T18:41:08.429 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:08.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-17T18:41:08.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-17T18:41:08.431 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836490 2026-02-17T18:41:08.431 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:41:08.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-17T18:41:08.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:41:09.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:41:09.658 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:41:09.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-17T18:41:09.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:41:10.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:41:10.886 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:41:11.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-17T18:41:11.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:11.122 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705674 2026-02-17T18:41:11.122 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:11.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:41:11.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-17T18:41:11.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:11.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705674 2026-02-17T18:41:11.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T18:41:11.125 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705674 2026-02-17T18:41:11.125 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:41:11.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T18:41:11.350 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:11.351 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542153 2026-02-17T18:41:11.351 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:11.352 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:41:11.352 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-17T18:41:11.352 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:11.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542153 2026-02-17T18:41:11.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-17T18:41:11.354 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 60129542153 2026-02-17T18:41:11.354 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:41:11.579 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-17T18:41:11.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:11.580 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411337 2026-02-17T18:41:11.580 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:11.581 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T18:41:11.581 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411337 2026-02-17T18:41:11.582 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:11.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-17T18:41:11.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411337' 2026-02-17T18:41:11.583 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.3 seq 77309411337 2026-02-17T18:41:11.583 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T18:41:11.806 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-17T18:41:11.807 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:41:11.807 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:11.807 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:12.123 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-17T18:41:12.123 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:41:12.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:41:12.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:41:12.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:41:12.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:41:12.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:41:12.124 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:41:12.347 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-17T18:41:12.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:41:12.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:12.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:12.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-17T18:41:12.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:41:12.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:41:12.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:274: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-17T18:41:12.884 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:12.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:275: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-17T18:41:13.106 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[2,3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:13.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:276: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-17T18:41:13.270 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:13.270 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T18:41:13.273 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:277: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-17T18:41:13.429 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:13.430 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T18:41:13.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-17T18:41:13.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: grep '[0,1]' 2026-02-17T18:41:13.659 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:13.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: ceph osd ok-to-stop 0 --max 20 2026-02-17T18:41:13.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: grep '[0,1]' 2026-02-17T18:41:13.880 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:13.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: ceph osd ok-to-stop 2 --max 2 2026-02-17T18:41:13.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: grep '[2,3]' 2026-02-17T18:41:14.107 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:14.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: ceph osd ok-to-stop 2 --max 20 2026-02-17T18:41:14.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: grep '[2,3]' 2026-02-17T18:41:14.329 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:14.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:285: TEST_0_osd: kill_daemons td/ok-to-stop TERM osd.0 2026-02-17T18:41:14.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:14.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:14.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:14.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:14.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:14.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:14.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:286: TEST_0_osd: ceph osd down 0 2026-02-17T18:41:14.662 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 is already down. 2026-02-17T18:41:14.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:287: TEST_0_osd: wait_for_peered 2026-02-17T18:41:14.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1731: wait_for_peered: local cmd= 2026-02-17T18:41:14.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1732: wait_for_peered: local num_peered=-1 2026-02-17T18:41:14.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1733: wait_for_peered: local cur_peered 2026-02-17T18:41:14.670 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: get_timeout_delays 90 .1 2026-02-17T18:41:14.671 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:41:14.671 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:41:14.671 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:41:14.671 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:41:14.671 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:41:14.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:41:14.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: local -a delays 2026-02-17T18:41:14.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1735: wait_for_peered: local -i loop=0 2026-02-17T18:41:14.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1737: wait_for_peered: flush_pg_stats 2026-02-17T18:41:14.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:41:14.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:41:14.980 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:41:14.980 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:41:14.981 INFO:tasks.workunit.client.0.trial068.stderr:2 2026-02-17T18:41:14.981 INFO:tasks.workunit.client.0.trial068.stderr:3' 2026-02-17T18:41:14.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:41:14.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:14.981 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:41:15.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-17T18:41:15.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-17T18:41:15.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-17T18:41:15.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-17T18:41:15.028 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:15.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:41:15.091 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705677 2026-02-17T18:41:15.091 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705677 2026-02-17T18:41:15.091 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-38654705677' 2026-02-17T18:41:15.092 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:15.092 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-17T18:41:15.153 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542157 2026-02-17T18:41:15.154 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542157 2026-02-17T18:41:15.154 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-38654705677 2-60129542157' 2026-02-17T18:41:15.154 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:41:15.154 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-17T18:41:15.216 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411340 2026-02-17T18:41:15.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411340 2026-02-17T18:41:15.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-38654705677 2-60129542157 3-77309411340' 2026-02-17T18:41:15.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:15.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705677 2026-02-17T18:41:15.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:15.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:41:15.218 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705677 2026-02-17T18:41:15.219 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:15.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705677 2026-02-17T18:41:15.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-17T18:41:15.220 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705677 2026-02-17T18:41:15.220 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:41:15.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705675 -lt 38654705677 2026-02-17T18:41:15.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:41:16.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:41:16.445 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:41:16.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705677 -lt 38654705677 2026-02-17T18:41:16.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:16.673 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542157 2026-02-17T18:41:16.673 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:16.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-17T18:41:16.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542157 2026-02-17T18:41:16.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:16.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542157 2026-02-17T18:41:16.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-17T18:41:16.675 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.2 seq 60129542157 2026-02-17T18:41:16.676 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T18:41:16.900 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-17T18:41:16.900 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:41:16.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411340 2026-02-17T18:41:16.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:41:16.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-17T18:41:16.902 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411340 2026-02-17T18:41:16.902 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:41:16.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411340 2026-02-17T18:41:16.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411340' 2026-02-17T18:41:16.904 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.3 seq 77309411340 2026-02-17T18:41:16.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T18:41:17.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411340 -lt 77309411340 2026-02-17T18:41:17.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: get_num_pgs 2026-02-17T18:41:17.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:17.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:17.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: test 32 == 0 2026-02-17T18:41:17.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1742: wait_for_peered: true 2026-02-17T18:41:17.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: get_num_active_or_peered 2026-02-17T18:41:17.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-17T18:41:17.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-17T18:41:17.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-17T18:41:17.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-17T18:41:17.442 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-17T18:41:17.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: cur_peered=32 2026-02-17T18:41:17.666 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: get_num_pgs 2026-02-17T18:41:17.666 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:41:17.666 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:41:17.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: test 32 = 32 2026-02-17T18:41:17.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: break 2026-02-17T18:41:17.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1762: wait_for_peered: return 0 2026-02-17T18:41:17.981 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:289: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-17T18:41:18.205 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":0,"num_not_ok_pgs":0} 2026-02-17T18:41:18.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:290: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-17T18:41:18.428 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:18.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:291: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-17T18:41:18.590 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:18.591 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T18:41:18.594 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:292: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-17T18:41:18.749 INFO:tasks.workunit.client.0.trial068.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-17T18:41:18.749 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-17T18:41:18.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:18.753 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:18.753 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:18.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:18.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:18.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:18.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:18.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T18:41:18.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:18.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:18.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:18.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:18.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:18.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:18.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:18.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:18.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:18.868 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:18.869 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:18.869 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:18.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:18.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:18.871 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:18.871 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:18.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:18.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:18.872 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:18.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:18.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:18.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:18.887 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:18.887 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:18.887 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:18.887 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:18.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:18.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:18.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:18.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:18.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:18.891 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:18.892 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:18.892 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:18.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:18.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:18.893 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:18.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:18.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:18.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:18.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:18.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:18.896 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:18.896 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:18.897 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:18.897 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:18.897 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:18.898 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:18.898 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:18.898 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:18.898 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T18:41:18.899 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:41:18.899 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:18.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:18.900 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39196 2026-02-17T18:41:18.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:41:18.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:41:18.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:41:18.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T18:41:18.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_1_mon_checks td/ok-to-stop 2026-02-17T18:41:18.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:28: TEST_1_mon_checks: local dir=td/ok-to-stop 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:30: TEST_1_mon_checks: CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:32: TEST_1_mon_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T18:41:18.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T18:41:18.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:18.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:18.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:18.920 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:18.921 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:18.921 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:18.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:18.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T18:41:18.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:18.944 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:41:18.944 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:18.944 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:18.944 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:18.946 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get fsid 2026-02-17T18:41:18.988 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:41:18.988 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:18.988 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:18.989 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:18.989 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:18.990 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:18.991 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get mon_host 2026-02-17T18:41:19.031 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:34: TEST_1_mon_checks: ceph mon ok-to-stop dne 2026-02-17T18:41:19.124 INFO:tasks.workunit.client.0.trial068.stderr:invalid mon(s) specified: [dne] 2026-02-17T18:41:19.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:35: TEST_1_mon_checks: ceph mon ok-to-stop a 2026-02-17T18:41:19.202 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a] 2026-02-17T18:41:19.204 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:37: TEST_1_mon_checks: ceph mon ok-to-add-offline 2026-02-17T18:41:19.280 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-17T18:41:19.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:39: TEST_1_mon_checks: ceph mon ok-to-rm a 2026-02-17T18:41:19.359 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-17T18:41:19.362 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:40: TEST_1_mon_checks: ceph mon ok-to-rm dne 2026-02-17T18:41:19.457 INFO:tasks.workunit.client.0.trial068.stderr:mon.dne does not exist 2026-02-17T18:41:19.463 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:19.464 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:19.464 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:19.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:19.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:19.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:19.570 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:19.571 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T18:41:19.571 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:19.571 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:19.571 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:19.571 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:19.571 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:19.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:19.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:19.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:19.576 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:19.576 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:19.577 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:19.577 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:19.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:19.578 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:19.579 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:19.579 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:19.579 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:19.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:19.580 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:19.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:19.581 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:19.581 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:19.583 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:19.583 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.583 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:19.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:19.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:19.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T18:41:19.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:19.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:19.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:19.587 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:19.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:19.588 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:19.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:19.589 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:19.589 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:19.589 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:19.590 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:19.590 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:19.590 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:19.591 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:19.592 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:19.592 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:19.593 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:19.593 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.593 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.593 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:19.594 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:19.594 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:19.594 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T18:41:19.595 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:41:19.595 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.596 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.596 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39196 2026-02-17T18:41:19.596 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:41:19.598 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:41:19.598 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:41:19.598 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T18:41:19.598 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_2_mons_checks td/ok-to-stop 2026-02-17T18:41:19.598 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:44: TEST_2_mons_checks: local dir=td/ok-to-stop 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:46: TEST_2_mons_checks: CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151 ' 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:48: TEST_2_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T18:41:19.599 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T18:41:19.615 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:19.616 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:19.616 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:19.616 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:19.616 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.616 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.616 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:19.616 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T18:41:19.640 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:19.641 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:41:19.641 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:19.641 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:19.641 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:19.642 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:19.643 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get fsid 2026-02-17T18:41:19.685 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:41:19.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:19.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:19.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:19.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:19.686 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:19.686 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:19.686 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:19.687 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:19.687 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.687 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.687 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:19.687 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:19.688 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get mon_host 2026-02-17T18:41:19.728 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:49: TEST_2_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T18:41:19.728 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:19.728 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:19.728 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T18:41:19.728 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:19.729 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T18:41:19.729 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T18:41:19.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:19.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:19.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:19.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:19.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:19.747 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T18:41:19.770 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:19.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T18:41:19.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:19.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:41:19.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:19.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:19.772 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:19.773 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get fsid 2026-02-17T18:41:19.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T18:41:19.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:19.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:41:19.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:19.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:19.816 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:41:19.816 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:41:19.817 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:41:19.817 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:19.817 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:19.817 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:19.817 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:41:19.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:19.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get mon_host 2026-02-17T18:41:19.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:51: TEST_2_mons_checks: ceph mon ok-to-stop dne 2026-02-17T18:41:19.952 INFO:tasks.workunit.client.0.trial068.stderr:invalid mon(s) specified: [dne] 2026-02-17T18:41:19.959 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:52: TEST_2_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:41:20.034 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (b) after stopping mons [a] 2026-02-17T18:41:20.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:53: TEST_2_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:41:20.109 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b] 2026-02-17T18:41:20.111 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:54: TEST_2_mons_checks: ceph mon ok-to-stop a b 2026-02-17T18:41:20.186 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b] 2026-02-17T18:41:20.189 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:56: TEST_2_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:41:20.282 INFO:tasks.workunit.client.0.trial068.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T18:41:20.288 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:58: TEST_2_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:41:20.381 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.a 2026-02-17T18:41:20.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:59: TEST_2_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:41:20.480 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.b 2026-02-17T18:41:20.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:60: TEST_2_mons_checks: ceph mon ok-to-rm dne 2026-02-17T18:41:20.578 INFO:tasks.workunit.client.0.trial068.stderr:mon.dne does not exist 2026-02-17T18:41:20.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:20.584 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:20.584 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:20.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:20.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:20.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:20.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:20.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T18:41:20.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:20.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:20.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:20.693 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:20.693 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:20.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:20.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:20.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:20.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:20.698 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:20.698 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:20.699 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:20.700 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:20.700 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:20.700 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:20.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:20.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:20.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:20.702 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:20.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:20.703 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:20.703 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:20.706 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:20.706 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.706 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:20.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:20.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:20.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:20.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:20.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:20.709 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:20.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:20.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:20.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:20.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:20.711 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:20.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:20.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:20.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:20.713 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:20.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:20.714 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:20.714 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:20.715 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:20.715 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.715 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.716 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:20.716 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:20.716 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:20.716 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T18:41:20.717 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:41:20.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.718 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39196 2026-02-17T18:41:20.719 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:41:20.720 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:41:20.720 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:41:20.720 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T18:41:20.720 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_3_mons_checks td/ok-to-stop 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:64: TEST_3_mons_checks: local dir=td/ok-to-stop 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:66: TEST_3_mons_checks: CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152 ' 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:68: TEST_3_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T18:41:20.721 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T18:41:20.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:20.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:20.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:20.737 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:20.737 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.737 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:20.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T18:41:20.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:20.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:41:20.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:20.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:20.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:20.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:20.763 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get fsid 2026-02-17T18:41:20.805 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:41:20.805 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:20.806 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:20.806 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:20.806 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:20.807 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:20.807 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:20.807 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:20.807 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:20.807 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.807 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.808 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:20.808 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:20.808 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get mon_host 2026-02-17T18:41:20.848 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:69: TEST_3_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T18:41:20.848 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:20.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:20.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T18:41:20.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:20.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T18:41:20.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T18:41:20.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:20.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:20.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:20.865 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:20.866 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.866 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.866 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:20.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T18:41:20.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:20.889 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T18:41:20.889 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:20.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:41:20.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:20.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:41:20.891 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:20.892 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get fsid 2026-02-17T18:41:20.934 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T18:41:20.934 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:20.934 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:41:20.934 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:20.934 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:20.934 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:41:20.934 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:41:20.935 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:41:20.935 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:20.935 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.935 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.935 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:41:20.935 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:20.936 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get mon_host 2026-02-17T18:41:20.976 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:70: TEST_3_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-17T18:41:20.977 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:20.977 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:20.977 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-17T18:41:20.977 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:20.977 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-17T18:41:20.977 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-17T18:41:20.994 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:20.994 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:20.994 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:20.994 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:20.994 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:20.994 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:20.994 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:20.995 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-17T18:41:21.016 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:21.017 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-17T18:41:21.017 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:21.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T18:41:21.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:21.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:21.019 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T18:41:21.019 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T18:41:21.019 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T18:41:21.019 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:21.019 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:21.019 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:21.019 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.c.asok 2026-02-17T18:41:21.020 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:21.020 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.c.asok config get fsid 2026-02-17T18:41:21.061 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-17T18:41:21.062 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:21.062 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T18:41:21.062 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:21.062 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:21.062 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T18:41:21.062 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T18:41:21.062 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T18:41:21.063 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:21.063 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:21.063 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:21.063 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.c.asok 2026-02-17T18:41:21.063 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:21.063 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.c.asok config get mon_host 2026-02-17T18:41:21.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:71: TEST_3_mons_checks: wait_for_quorum 60 3 2026-02-17T18:41:21.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T18:41:21.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-17T18:41:21.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T18:41:21.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-17T18:41:21.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T18:41:21.106 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T18:41:21.106 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771353741 2026-02-17T18:41:21.106 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:21.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353681 -lt 1771353741 ]] 2026-02-17T18:41:21.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:41:21.107 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:21.808 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:21.810 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:21.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:21.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:41:21.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T18:41:21.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T18:41:21.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T18:41:21.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T18:41:21.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T18:41:21.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:73: TEST_3_mons_checks: ceph mon ok-to-stop dne 2026-02-17T18:41:21.904 INFO:tasks.workunit.client.0.trial068.stderr:invalid mon(s) specified: [dne] 2026-02-17T18:41:21.909 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:74: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:41:21.999 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c) after stopping [a] 2026-02-17T18:41:22.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:75: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:41:22.096 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-17T18:41:22.101 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:76: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-17T18:41:22.193 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,b) after stopping [c] 2026-02-17T18:41:22.198 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:77: TEST_3_mons_checks: ceph mon ok-to-stop a b 2026-02-17T18:41:22.270 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a,b] 2026-02-17T18:41:22.272 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:78: TEST_3_mons_checks: ceph mon ok-to-stop b c 2026-02-17T18:41:22.345 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b,c] 2026-02-17T18:41:22.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:79: TEST_3_mons_checks: ceph mon ok-to-stop a b c 2026-02-17T18:41:22.418 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b,c] 2026-02-17T18:41:22.420 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:81: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:41:22.513 INFO:tasks.workunit.client.0.trial068.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T18:41:22.518 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:83: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:41:22.611 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.a 2026-02-17T18:41:22.616 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:84: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:41:22.707 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.b 2026-02-17T18:41:22.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:85: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-17T18:41:22.804 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.c 2026-02-17T18:41:22.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:87: TEST_3_mons_checks: kill_daemons td/ok-to-stop KILL mon.b 2026-02-17T18:41:22.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:22.811 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:22.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:22.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:22.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:22.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:22.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:88: TEST_3_mons_checks: wait_for_quorum 60 2 2026-02-17T18:41:22.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T18:41:22.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=2 2026-02-17T18:41:22.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T18:41:22.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 2 ]] 2026-02-17T18:41:22.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T18:41:22.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T18:41:22.918 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771353742 2026-02-17T18:41:22.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:22.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353682 -lt 1771353742 ]] 2026-02-17T18:41:22.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:22.919 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:26.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:26.021 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:26.021 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:26.021 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:26.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:26.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:26.023 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:26.024 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353686 -lt 1771353742 ]] 2026-02-17T18:41:26.024 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:26.024 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:26.123 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:26.123 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:26.124 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:26.124 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:26.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:26.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:26.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:26.127 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353686 -lt 1771353742 ]] 2026-02-17T18:41:26.127 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:26.127 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.227 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.228 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.228 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.230 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.231 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.231 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.327 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.328 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.328 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.328 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.431 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.434 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.534 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.535 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.535 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.537 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.538 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.538 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.637 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.637 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.638 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.639 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.639 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.640 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.641 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.641 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.641 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.739 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.739 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.740 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.740 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.742 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.743 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.743 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.840 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.840 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.841 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.841 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.842 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.842 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.843 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.843 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.844 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:29.942 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:29.943 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.943 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:29.943 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:29.945 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:29.945 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:29.946 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:29.947 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353689 -lt 1771353742 ]] 2026-02-17T18:41:29.947 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:29.947 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:30.044 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:30.044 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:30.045 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:30.045 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:30.046 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:30.046 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:30.047 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:30.048 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353690 -lt 1771353742 ]] 2026-02-17T18:41:30.048 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-17T18:41:30.048 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:42.149 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:42.150 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:42.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-17T18:41:42.151 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:20.729926Z","created":"2026-02-17T18:41:20.729926Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729920+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729924+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:20:729925+0000","crush_location":[]}]}}' 2026-02-17T18:41:42.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T18:41:42.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T18:41:42.153 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T18:41:42.153 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T18:41:42.153 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T18:41:42.153 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:90: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:41:42.226 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a] 2026-02-17T18:41:42.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:91: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:41:42.320 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-17T18:41:42.325 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:92: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-17T18:41:42.397 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [c] 2026-02-17T18:41:42.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:94: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:41:42.472 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-17T18:41:42.474 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:96: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:41:42.547 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-17T18:41:42.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:97: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:41:42.643 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.b 2026-02-17T18:41:42.648 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:98: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-17T18:41:42.723 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-17T18:41:42.725 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:42.726 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:42.726 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:42.726 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:42.726 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:42.726 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:42.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:42.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T18:41:42.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:42.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:42.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:42.834 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:42.834 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:42.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:42.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:42.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:42.839 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:42.839 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:42.840 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:42.840 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:42.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:42.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:42.841 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:42.842 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:42.842 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:42.843 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:42.843 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:42.843 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:42.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:42.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:42.848 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:42.848 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:42.848 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:42.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:42.850 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:42.850 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:42.850 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:42.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:42.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:42.852 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:41:42.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:41:42.853 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:41:42.854 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:41:42.854 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:41:42.854 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:41:42.855 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:42.855 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:41:42.855 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:41:42.856 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:41:42.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:41:42.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:41:42.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:41:42.858 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:41:42.858 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:42.858 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:42.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:41:42.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:41:42.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:41:42.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T18:41:42.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:41:42.861 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:42.861 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:42.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39196 2026-02-17T18:41:42.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:41:42.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:41:42.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:41:42.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T18:41:42.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_4_mons_checks td/ok-to-stop 2026-02-17T18:41:42.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:102: TEST_4_mons_checks: local dir=td/ok-to-stop 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:104: TEST_4_mons_checks: CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153 ' 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:106: TEST_4_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T18:41:42.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T18:41:42.880 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:42.880 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:42.880 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:42.881 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:42.881 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:42.881 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:42.881 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:42.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T18:41:42.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:42.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:41:42.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:42.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:42.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:42.906 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get fsid 2026-02-17T18:41:42.947 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:42.948 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:41:42.949 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:42.949 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get mon_host 2026-02-17T18:41:42.989 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:107: TEST_4_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T18:41:42.990 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:42.990 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:42.990 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T18:41:42.990 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:42.990 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T18:41:42.990 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T18:41:43.006 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:43.006 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:43.006 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:43.007 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:43.007 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.007 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.007 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:43.007 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T18:41:43.029 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:43.030 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T18:41:43.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:43.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:41:43.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:43.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:43.032 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:41:43.032 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:41:43.032 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:41:43.033 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:43.033 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.033 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.033 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:41:43.033 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:43.033 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get fsid 2026-02-17T18:41:43.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T18:41:43.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:43.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:41:43.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:43.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:43.074 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:41:43.074 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:41:43.075 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:41:43.075 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:43.075 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.075 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.075 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:41:43.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:43.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get mon_host 2026-02-17T18:41:43.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:108: TEST_4_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-17T18:41:43.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:43.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:43.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-17T18:41:43.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:43.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-17T18:41:43.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-17T18:41:43.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:43.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:43.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:43.133 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:43.133 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.133 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.134 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:43.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-17T18:41:43.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:43.158 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-17T18:41:43.158 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:43.158 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T18:41:43.158 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:43.158 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:43.159 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T18:41:43.159 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T18:41:43.159 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T18:41:43.159 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:43.160 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.160 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.160 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.c.asok 2026-02-17T18:41:43.160 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:43.160 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.c.asok config get fsid 2026-02-17T18:41:43.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-17T18:41:43.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:43.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T18:41:43.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:43.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:43.201 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T18:41:43.201 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T18:41:43.201 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T18:41:43.201 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:43.202 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.202 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.202 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.c.asok 2026-02-17T18:41:43.202 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:43.202 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.c.asok config get mon_host 2026-02-17T18:41:43.243 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:109: TEST_4_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-17T18:41:43.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:41:43.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:41:43.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-17T18:41:43.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:41:43.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-17T18:41:43.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-17T18:41:43.261 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:41:43.262 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:41:43.262 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:41:43.262 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:41:43.262 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.262 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.262 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:41:43.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-17T18:41:43.285 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:41:43.286 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-17T18:41:43.286 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:43.287 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T18:41:43.287 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:41:43.287 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:41:43.288 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T18:41:43.288 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T18:41:43.288 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T18:41:43.288 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:43.288 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.288 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.288 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.d.asok 2026-02-17T18:41:43.289 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:43.289 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.d.asok config get fsid 2026-02-17T18:41:43.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-17T18:41:43.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:41:43.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T18:41:43.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:41:43.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:41:43.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T18:41:43.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T18:41:43.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T18:41:43.332 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:41:43.332 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:41:43.332 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:41:43.332 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.d.asok 2026-02-17T18:41:43.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:41:43.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.d.asok config get mon_host 2026-02-17T18:41:43.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:110: TEST_4_mons_checks: wait_for_quorum 60 4 2026-02-17T18:41:43.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T18:41:43.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-17T18:41:43.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T18:41:43.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-17T18:41:43.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T18:41:43.374 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T18:41:43.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771353763 2026-02-17T18:41:43.375 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:43.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353703 -lt 1771353763 ]] 2026-02-17T18:41:43.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:41:43.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:43.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:43.679 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:43.680 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:43.680 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:41:43.682 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T18:41:43.682 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T18:41:43.682 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T18:41:43.682 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T18:41:43.683 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T18:41:43.683 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:112: TEST_4_mons_checks: ceph mon ok-to-stop dne 2026-02-17T18:41:43.775 INFO:tasks.workunit.client.0.trial068.stderr:invalid mon(s) specified: [dne] 2026-02-17T18:41:43.780 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:113: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:41:43.871 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-17T18:41:43.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:114: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:41:46.970 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,c,d) after stopping [b] 2026-02-17T18:41:46.976 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:115: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-17T18:41:50.069 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,b,d) after stopping [c] 2026-02-17T18:41:50.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:116: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-17T18:41:50.164 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,b,c) after stopping [d] 2026-02-17T18:41:50.169 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:117: TEST_4_mons_checks: ceph mon ok-to-stop a b 2026-02-17T18:41:50.241 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [a,b] 2026-02-17T18:41:50.243 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:118: TEST_4_mons_checks: ceph mon ok-to-stop c d 2026-02-17T18:41:50.315 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (a,b) after stopping mons [c,d] 2026-02-17T18:41:50.317 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:120: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:41:50.408 INFO:tasks.workunit.client.0.trial068.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T18:41:50.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:122: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:41:50.506 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.a 2026-02-17T18:41:50.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:123: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:41:50.605 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.b 2026-02-17T18:41:50.611 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:124: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-17T18:41:50.701 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.c 2026-02-17T18:41:50.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:126: TEST_4_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-17T18:41:50.707 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:41:50.707 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:41:50.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:41:50.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:41:50.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:41:50.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:41:50.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:127: TEST_4_mons_checks: wait_for_quorum 60 3 2026-02-17T18:41:50.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T18:41:50.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-17T18:41:50.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T18:41:50.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-17T18:41:50.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T18:41:50.814 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T18:41:50.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771353770 2026-02-17T18:41:50.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:50.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353710 -lt 1771353770 ]] 2026-02-17T18:41:50.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:41:50.816 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:50.913 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:50.913 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:50.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:50.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:41:50.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:50.916 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:50.916 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:50.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353710 -lt 1771353770 ]] 2026-02-17T18:41:50.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:41:50.917 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:54.015 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:54.016 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:54.016 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:54.017 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:41:54.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:54.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:54.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:54.019 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353714 -lt 1771353770 ]] 2026-02-17T18:41:54.019 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:41:54.019 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:54.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:54.118 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:54.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:54.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:41:54.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:54.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:54.121 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:54.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353714 -lt 1771353770 ]] 2026-02-17T18:41:54.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:41:54.122 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:41:57.222 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:41:57.223 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:57.223 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:41:57.223 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:41:57.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:41:57.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:41:57.226 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:41:57.226 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353717 -lt 1771353770 ]] 2026-02-17T18:41:57.226 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:41:57.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:00.329 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:00.330 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:42:00.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:42:00.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:00.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:00.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:00.332 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:00.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353720 -lt 1771353770 ]] 2026-02-17T18:42:00.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:00.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:06.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:06.440 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:42:06.440 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:41:42.873517Z","created":"2026-02-17T18:41:42.873517Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873511+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873514+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873515+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:41:42:873516+0000","crush_location":[]}]}}' 2026-02-17T18:42:06.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:06.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T18:42:06.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T18:42:06.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T18:42:06.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T18:42:06.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T18:42:06.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:129: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:42:09.538 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-17T18:42:09.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:130: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:42:09.617 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-17T18:42:09.619 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:131: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-17T18:42:12.693 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-17T18:42:12.695 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:132: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-17T18:42:12.769 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-17T18:42:12.772 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:134: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:42:12.865 INFO:tasks.workunit.client.0.trial068.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T18:42:12.871 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:136: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:42:12.965 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.a 2026-02-17T18:42:12.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:137: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:42:13.065 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.b 2026-02-17T18:42:13.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:138: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-17T18:42:13.163 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.c 2026-02-17T18:42:13.169 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:139: TEST_4_mons_checks: ceph mon ok-to-rm d 2026-02-17T18:42:13.264 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.d 2026-02-17T18:42:13.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T18:42:13.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:42:13.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:42:13.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:42:13.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:42:13.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:42:13.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:42:13.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T18:42:13.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:42:13.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:42:13.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:42:13.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:42:13.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:42:13.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:42:13.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:42:13.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:42:13.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:42:13.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:42:13.386 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:42:13.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:42:13.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:42:13.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:42:13.387 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:42:13.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:42:13.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:42:13.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:42:13.389 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:42:13.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:42:13.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:42:13.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:42:13.395 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:42:13.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:42:13.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:42:13.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:42:13.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:42:13.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:42:13.399 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:42:13.400 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:42:13.400 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:42:13.401 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:42:13.401 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:42:13.401 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:42:13.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:42:13.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:42:13.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:42:13.403 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:42:13.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:42:13.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:42:13.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:42:13.405 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:42:13.406 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.406 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:42:13.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:42:13.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:42:13.407 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-17T18:42:13.408 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:42:13.408 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.408 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.39196 2026-02-17T18:42:13.409 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:42:13.410 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_5_mons_checks td/ok-to-stop 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:143: TEST_5_mons_checks: local dir=td/ok-to-stop 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:145: TEST_5_mons_checks: CEPH_ARGS='--fsid=086dd9a9-4d8b-4115-9011-95ea227990b6 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153,127.0.0.1:7154 ' 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:147: TEST_5_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:42:13.411 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-17T18:42:13.412 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-17T18:42:13.429 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:42:13.429 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:42:13.429 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:42:13.429 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:42:13.430 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.430 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.430 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:42:13.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-17T18:42:13.453 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:42:13.454 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:42:13.454 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.454 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:42:13.454 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:42:13.455 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.456 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get fsid 2026-02-17T18:42:13.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:42:13.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.a.asok 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.a.asok config get mon_host 2026-02-17T18:42:13.546 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:148: TEST_5_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-17T18:42:13.546 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:42:13.546 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:42:13.546 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-17T18:42:13.546 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:42:13.547 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-17T18:42:13.547 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-17T18:42:13.563 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:42:13.564 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:42:13.564 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:42:13.564 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:42:13.564 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.564 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.564 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:42:13.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-17T18:42:13.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:42:13.588 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-17T18:42:13.588 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.589 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:42:13.589 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:42:13.589 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:42:13.590 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:42:13.590 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:42:13.590 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:42:13.590 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.591 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.591 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.591 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:42:13.591 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.591 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get fsid 2026-02-17T18:42:13.635 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-17T18:42:13.635 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.636 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.b.asok 2026-02-17T18:42:13.637 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.637 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.b.asok config get mon_host 2026-02-17T18:42:13.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:149: TEST_5_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-17T18:42:13.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:42:13.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:42:13.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-17T18:42:13.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:42:13.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-17T18:42:13.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-17T18:42:13.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:42:13.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:42:13.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:42:13.696 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:42:13.696 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.696 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.697 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:42:13.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-17T18:42:13.720 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:42:13.721 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-17T18:42:13.721 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.722 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T18:42:13.722 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:42:13.722 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:42:13.723 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T18:42:13.723 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T18:42:13.723 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T18:42:13.723 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.723 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.724 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.724 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.c.asok 2026-02-17T18:42:13.724 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.724 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.c.asok config get fsid 2026-02-17T18:42:13.768 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-17T18:42:13.768 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.768 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.c.asok 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.769 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.c.asok config get mon_host 2026-02-17T18:42:13.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:150: TEST_5_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-17T18:42:13.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:42:13.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:42:13.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-17T18:42:13.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:42:13.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-17T18:42:13.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-17T18:42:13.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:42:13.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:42:13.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:42:13.830 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:42:13.830 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.830 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.830 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:42:13.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-17T18:42:13.854 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:42:13.855 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-17T18:42:13.855 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.855 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T18:42:13.855 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:42:13.855 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:42:13.855 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T18:42:13.856 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T18:42:13.856 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T18:42:13.857 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.857 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.857 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.857 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.d.asok 2026-02-17T18:42:13.857 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.857 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.d.asok config get fsid 2026-02-17T18:42:13.899 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.900 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.d.asok 2026-02-17T18:42:13.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.d.asok config get mon_host 2026-02-17T18:42:13.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:151: TEST_5_mons_checks: run_mon td/ok-to-stop e --public-addr=127.0.0.1:7154 2026-02-17T18:42:13.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-17T18:42:13.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:42:13.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=e 2026-02-17T18:42:13.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:42:13.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/e 2026-02-17T18:42:13.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id e --mkfs --mon-data=td/ok-to-stop/e --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7154 2026-02-17T18:42:13.960 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:42:13.960 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:42:13.960 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:42:13.960 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:42:13.960 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.961 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.961 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.39196/$cluster-$name.asok' 2026-02-17T18:42:13.961 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id e --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/e '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.39196/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7154 2026-02-17T18:42:13.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:42:13.984 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e fsid 2026-02-17T18:42:13.984 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:13.984 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-17T18:42:13.985 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:42:13.985 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:42:13.986 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-17T18:42:13.986 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-17T18:42:13.986 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-17T18:42:13.986 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:13.986 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:13.986 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:13.986 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.e.asok 2026-02-17T18:42:13.987 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:13.987 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.e.asok config get fsid 2026-02-17T18:42:14.030 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e mon_host 2026-02-17T18:42:14.030 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:42:14.031 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:42:14.032 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.39196/ceph-mon.e.asok 2026-02-17T18:42:14.032 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:42:14.032 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.39196/ceph-mon.e.asok config get mon_host 2026-02-17T18:42:14.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:152: TEST_5_mons_checks: wait_for_quorum 60 5 2026-02-17T18:42:14.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T18:42:14.079 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=5 2026-02-17T18:42:14.079 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T18:42:14.079 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 5 ]] 2026-02-17T18:42:14.079 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T18:42:14.079 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T18:42:14.080 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771353794 2026-02-17T18:42:14.080 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:14.080 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353734 -lt 1771353794 ]] 2026-02-17T18:42:14.080 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-17T18:42:14.081 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:20.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:20.184 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:20.184 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-17T18:42:20.185 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:20.186 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T18:42:20.186 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T18:42:20.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T18:42:20.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T18:42:20.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T18:42:20.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:154: TEST_5_mons_checks: ceph mon ok-to-stop dne 2026-02-17T18:42:20.280 INFO:tasks.workunit.client.0.trial068.stderr:invalid mon(s) specified: [dne] 2026-02-17T18:42:20.285 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:155: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:42:20.376 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-17T18:42:20.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:156: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:42:20.472 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,c,d,e) after stopping [b] 2026-02-17T18:42:20.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:157: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-17T18:42:20.569 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,b,d,e) after stopping [c] 2026-02-17T18:42:20.574 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:158: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-17T18:42:20.668 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,b,c,e) after stopping [d] 2026-02-17T18:42:20.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:159: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-17T18:42:20.766 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,b,c,d) after stopping [e] 2026-02-17T18:42:20.771 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:160: TEST_5_mons_checks: ceph mon ok-to-stop a b 2026-02-17T18:42:20.867 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (c,d,e) after stopping [a,b] 2026-02-17T18:42:20.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:161: TEST_5_mons_checks: ceph mon ok-to-stop c d 2026-02-17T18:42:20.966 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (a,b,e) after stopping [c,d] 2026-02-17T18:42:20.971 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:162: TEST_5_mons_checks: ceph mon ok-to-stop a b c 2026-02-17T18:42:21.047 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (d,e) after stopping mons [a,b,c] 2026-02-17T18:42:21.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:164: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:42:21.144 INFO:tasks.workunit.client.0.trial068.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T18:42:21.149 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:166: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:42:21.240 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.a 2026-02-17T18:42:21.246 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:167: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:42:21.339 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.b 2026-02-17T18:42:21.345 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:168: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-17T18:42:21.438 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.c 2026-02-17T18:42:21.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:169: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-17T18:42:21.538 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.d 2026-02-17T18:42:21.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:170: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-17T18:42:21.640 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.e 2026-02-17T18:42:21.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:172: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-17T18:42:21.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:42:21.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:42:21.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:42:21.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:42:21.647 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:42:21.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:42:21.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:173: TEST_5_mons_checks: wait_for_quorum 60 4 2026-02-17T18:42:21.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T18:42:21.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-17T18:42:21.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T18:42:21.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-17T18:42:21.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T18:42:21.753 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T18:42:21.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771353801 2026-02-17T18:42:21.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:21.755 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353741 -lt 1771353801 ]] 2026-02-17T18:42:21.755 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:21.755 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:21.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:21.858 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:21.859 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:21.859 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:21.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:21.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:21.861 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:21.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353741 -lt 1771353801 ]] 2026-02-17T18:42:21.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:21.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:21.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:21.962 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:21.963 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:21.963 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:21.965 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:21.965 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:21.965 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:21.966 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353741 -lt 1771353801 ]] 2026-02-17T18:42:21.966 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:21.967 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.064 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.065 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.065 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.065 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.067 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.067 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.068 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.068 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.069 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.168 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.170 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.171 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.274 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.275 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.275 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.277 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.278 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.375 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.379 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.476 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.477 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.477 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.479 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.480 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.580 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.581 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.581 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.583 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.584 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.584 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.584 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.683 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.684 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.685 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.686 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.686 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.688 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.788 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.789 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.790 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.791 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.891 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.892 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.892 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.895 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:22.993 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:22.993 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.994 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:22.994 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:22.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:22.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:22.996 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:22.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353742 -lt 1771353801 ]] 2026-02-17T18:42:22.997 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:22.997 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:23.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:23.096 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:23.096 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:23.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:23.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:23.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:23.099 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:23.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353743 -lt 1771353801 ]] 2026-02-17T18:42:23.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:23.100 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:23.196 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:23.196 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:23.196 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:23.197 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:23.199 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:23.199 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:23.199 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:23.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353743 -lt 1771353801 ]] 2026-02-17T18:42:23.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:23.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:23.299 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:23.299 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:23.300 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:23.300 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:23.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:23.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:23.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:23.303 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353743 -lt 1771353801 ]] 2026-02-17T18:42:23.303 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:23.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:26.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:26.404 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.405 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:26.405 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.407 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:26.407 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:26.407 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:26.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353746 -lt 1771353801 ]] 2026-02-17T18:42:26.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:26.408 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:26.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:26.507 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.507 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.507 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:26.509 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:26.509 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:26.510 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:26.510 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353746 -lt 1771353801 ]] 2026-02-17T18:42:26.510 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:26.511 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:26.610 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:26.610 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.611 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.611 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:26.613 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:26.613 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:26.614 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:26.614 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353746 -lt 1771353801 ]] 2026-02-17T18:42:26.614 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:26.615 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:26.714 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:26.714 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.714 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.715 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:26.716 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:26.717 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:26.717 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:26.717 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353746 -lt 1771353801 ]] 2026-02-17T18:42:26.717 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:26.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:26.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:26.816 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:26.818 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:26.818 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:26.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:26.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353746 -lt 1771353801 ]] 2026-02-17T18:42:26.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:26.820 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:26.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:26.922 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.923 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:26.923 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:26.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:26.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:26.925 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:26.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353746 -lt 1771353801 ]] 2026-02-17T18:42:26.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:26.926 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:30.028 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:30.028 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:30.029 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:30.029 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:30.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:30.030 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:30.031 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:30.032 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353750 -lt 1771353801 ]] 2026-02-17T18:42:30.032 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:30.032 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:30.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:30.134 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:30.134 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:30.134 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:30.136 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:30.136 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:30.137 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:30.137 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353750 -lt 1771353801 ]] 2026-02-17T18:42:30.138 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:30.138 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:30.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:30.236 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:30.236 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:30.236 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:30.238 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:30.238 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:30.239 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:30.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353750 -lt 1771353801 ]] 2026-02-17T18:42:30.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-17T18:42:30.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:36.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:36.941 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:36.941 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:36.942 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-17T18:42:36.944 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T18:42:36.944 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T18:42:36.944 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T18:42:36.944 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T18:42:36.944 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T18:42:36.944 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:175: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:42:40.037 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-17T18:42:40.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:176: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:42:40.133 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (c,d,e) after stopping [b] 2026-02-17T18:42:40.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:177: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-17T18:42:43.230 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,d,e) after stopping [c] 2026-02-17T18:42:43.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:178: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-17T18:42:43.328 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,e) after stopping [d] 2026-02-17T18:42:43.335 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:179: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-17T18:42:43.426 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-17T18:42:43.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:181: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:42:43.524 INFO:tasks.workunit.client.0.trial068.stderr:adding another mon that is not yet online will not break quorum 2026-02-17T18:42:43.529 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:183: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:42:43.621 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.a 2026-02-17T18:42:43.626 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:184: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:42:46.720 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.b 2026-02-17T18:42:46.726 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:185: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-17T18:42:49.820 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.c 2026-02-17T18:42:49.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:186: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-17T18:42:49.916 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.d 2026-02-17T18:42:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:187: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-17T18:42:50.012 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.e 2026-02-17T18:42:50.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:189: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.e 2026-02-17T18:42:50.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:42:50.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:42:50.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:42:50.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:42:50.018 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:42:50.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:42:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:190: TEST_5_mons_checks: wait_for_quorum 60 3 2026-02-17T18:42:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-17T18:42:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-17T18:42:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-17T18:42:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-17T18:42:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-17T18:42:50.125 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-17T18:42:50.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771353830 2026-02-17T18:42:50.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:50.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353770 -lt 1771353830 ]] 2026-02-17T18:42:50.127 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:50.127 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:50.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:50.227 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:50.228 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:50.228 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:50.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:50.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:50.230 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:50.231 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353770 -lt 1771353830 ]] 2026-02-17T18:42:50.231 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:50.231 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:50.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:50.330 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:50.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:50.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:50.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:50.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:50.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:50.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353770 -lt 1771353830 ]] 2026-02-17T18:42:50.334 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:50.334 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:50.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:50.434 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:50.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:50.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:50.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:50.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:50.437 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:50.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353770 -lt 1771353830 ]] 2026-02-17T18:42:50.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:50.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:53.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:53.538 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:53.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:53.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:53.540 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:53.540 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:53.540 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:53.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353773 -lt 1771353830 ]] 2026-02-17T18:42:53.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:53.541 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:56.643 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:56.643 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.644 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:56.644 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:56.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:56.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:56.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353776 -lt 1771353830 ]] 2026-02-17T18:42:56.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:56.647 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:56.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:56.749 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.750 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.750 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:56.751 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:56.751 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:56.752 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:56.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353776 -lt 1771353830 ]] 2026-02-17T18:42:56.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:56.753 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:56.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:56.849 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.850 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.850 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:56.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:56.852 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:56.852 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:56.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353776 -lt 1771353830 ]] 2026-02-17T18:42:56.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:56.854 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:56.954 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:56.954 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:56.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:56.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:56.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:56.957 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:56.958 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353776 -lt 1771353830 ]] 2026-02-17T18:42:56.958 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:56.958 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:42:57.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:42:57.055 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:57.056 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":20,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:42:57.056 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:42:57.058 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-17T18:42:57.058 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-17T18:42:57.058 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-17T18:42:57.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771353777 -lt 1771353830 ]] 2026-02-17T18:42:57.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-17T18:42:57.059 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-17T18:43:06.164 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-17T18:43:06.164 INFO:tasks.workunit.client.0.trial068.stderr:{"election_epoch":16,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:43:06.165 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":16,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"086dd9a9-4d8b-4115-9011-95ea227990b6","modified":"2026-02-17T18:42:13.421518Z","created":"2026-02-17T18:42:13.421518Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421509+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421513+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421514+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421515+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-17T18:42:13:421516+0000","crush_location":[]}]}}' 2026-02-17T18:43:06.165 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-17T18:43:06.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-17T18:43:06.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-17T18:43:06.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-17T18:43:06.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-17T18:43:06.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-17T18:43:06.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:192: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-17T18:43:09.263 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-17T18:43:09.269 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:193: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-17T18:43:09.346 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-17T18:43:09.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:194: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-17T18:43:09.424 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-17T18:43:09.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:195: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-17T18:43:09.502 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-17T18:43:09.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:196: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-17T18:43:12.598 INFO:tasks.workunit.client.0.trial068.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-17T18:43:12.604 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:198: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-17T18:43:15.679 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-17T18:43:15.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:200: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-17T18:43:15.774 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.a 2026-02-17T18:43:15.780 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:201: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-17T18:43:15.855 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: removing mon.b would break quorum 2026-02-17T18:43:15.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:202: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-17T18:43:18.932 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-17T18:43:18.934 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:203: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-17T18:43:19.008 INFO:tasks.workunit.client.0.trial068.stderr:Error EBUSY: removing mon.d would break quorum 2026-02-17T18:43:19.010 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:204: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-17T18:43:19.103 INFO:tasks.workunit.client.0.trial068.stderr:safe to remove mon.e 2026-02-17T18:43:19.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-17T18:43:19.110 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:19.110 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:19.110 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:19.110 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:19.110 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:19.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:19.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-17T18:43:19.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:43:19.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:19.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:43:19.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:19.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:19.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:19.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:19.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:19.223 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:19.223 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:19.224 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:19.225 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:19.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:19.226 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:19.226 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:19.226 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:19.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:19.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:19.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:19.228 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:19.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:19.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:43:19.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:19.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:43:19.240 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/ok-to-stop 0 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-17T18:43:19.242 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:19.243 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:19.243 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:19.243 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:19.243 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:19.243 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:19.244 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:19.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:19.244 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:19.245 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:19.245 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:19.246 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:19.246 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:19.246 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:19.247 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:19.247 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:19.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:19.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T18:43:19.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-17T18:43:19.250 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:19.250 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.250 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.39196 2026-02-17T18:43:19.250 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.39196 2026-02-17T18:43:19.251 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:19.251 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:19.251 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T18:43:19.252 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T18:43:19.252 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T18:43:19.318 INFO:tasks.workunit:Running workunit misc/rados-striper.sh... 2026-02-17T18:43:19.319 DEBUG:teuthology.orchestra.run.trial068:workunit test misc/rados-striper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh 2026-02-17T18:43:19.381 INFO:tasks.workunit.client.0.trial068.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T18:43:19.384 INFO:tasks.workunit.client.0.trial068.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/rados-striper 2026-02-17T18:43:19.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:20: run: local dir=td/rados-striper 2026-02-17T18:43:19.386 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:21: run: shift 2026-02-17T18:43:19.386 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: export CEPH_MON=127.0.0.1:7116 2026-02-17T18:43:19.386 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: CEPH_MON=127.0.0.1:7116 2026-02-17T18:43:19.386 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:24: run: export CEPH_ARGS 2026-02-17T18:43:19.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: uuidgen 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: CEPH_ARGS+='--fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none ' 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7116 ' 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:29: run: setup td/rados-striper 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/rados-striper 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/rados-striper 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:19.388 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:19.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:19.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:19.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:19.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:19.389 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:19.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:19.390 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:19.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:19.390 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:19.390 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:19.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:19.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:19.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:19.391 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:19.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:19.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:19.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-17T18:43:19.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:19.394 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.394 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:19.394 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.54356 2026-02-17T18:43:19.394 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:19.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:19.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/rados-striper 2026-02-17T18:43:19.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:19.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:19.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.54356 2026-02-17T18:43:19.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/rados-striper 1' TERM HUP INT 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:32: run: run_mon td/rados-striper a 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/rados-striper 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:19.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:19.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:19.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/rados-striper/a 2026-02-17T18:43:19.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/rados-striper/a --run-dir=td/rados-striper 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.54356/$cluster-$name.asok' 2026-02-17T18:43:19.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/rados-striper/a '--log-file=td/rados-striper/$name.log' '--admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' --mon-cluster-log-file=td/rados-striper/log --run-dir=td/rados-striper '--pid-file=td/rados-striper/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:19.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:19.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:19.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:19.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:19.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:19.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:19.439 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:19.439 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:19.440 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:19.441 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:19.441 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.441 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:19.441 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.54356/ceph-mon.a.asok 2026-02-17T18:43:19.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:19.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.54356/ceph-mon.a.asok config get fsid 2026-02-17T18:43:19.483 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:19.483 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:19.483 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:19.483 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:19.483 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.54356/ceph-mon.a.asok 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:19.484 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.54356/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:33: run: run_osd td/rados-striper 0 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/0 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:19.531 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/0' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/0/journal' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.54356/$cluster-$name.asok' 2026-02-17T18:43:19.532 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/0 2026-02-17T18:43:19.533 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:19.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=02f92415-04dc-48ea-a1f6-6d4335c1ad7a 2026-02-17T18:43:19.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 02f92415-04dc-48ea-a1f6-6d4335c1ad7a' 2026-02-17T18:43:19.534 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 02f92415-04dc-48ea-a1f6-6d4335c1ad7a 2026-02-17T18:43:19.534 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:19.539 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDHtpRpG1YXIBAAdMNThp1EOO5+JyxS8hotOg== 2026-02-17T18:43:19.539 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDHtpRpG1YXIBAAdMNThp1EOO5+JyxS8hotOg=="}' 2026-02-17T18:43:19.539 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 02f92415-04dc-48ea-a1f6-6d4335c1ad7a -i td/rados-striper/0/new.json 2026-02-17T18:43:19.637 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:19.642 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/0/new.json 2026-02-17T18:43:19.643 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDHtpRpG1YXIBAAdMNThp1EOO5+JyxS8hotOg== --osd-uuid 02f92415-04dc-48ea-a1f6-6d4335c1ad7a 2026-02-17T18:43:19.658 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:19.657+0000 7f3b00c31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:19.659 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:19.658+0000 7f3b00c31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:19.660 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:19.659+0000 7f3b00c31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:19.661 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:19.659+0000 7f3b00c31780 -1 bdev(0x3f22c00 td/rados-striper/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:19.661 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:19.659+0000 7f3b00c31780 -1 bluestore(td/rados-striper/0) _read_fsid unparsable uuid 2026-02-17T18:43:19.989 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/0/keyring 2026-02-17T18:43:19.989 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:19.989 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:19.990 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:19.990 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:20.087 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:20.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:20.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:20.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:20.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:20.088 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:20.100 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:20.099+0000 7fbfe2029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:20.101 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:20.100+0000 7fbfe2029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:20.102 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:20.101+0000 7fbfe2029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:20.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:20.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:20.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:20.182 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:20.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:20.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:20.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:20.183 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:20.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:20.184 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:20.273 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:20.272+0000 7fbfe2029780 -1 Falling back to public interface 2026-02-17T18:43:20.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:20.546 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:20.545+0000 7fbfe2029780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:21.275 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:21.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:21.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:21.276 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:21.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:21.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:21.370 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/986443870,v1:127.0.0.1:6801/986443870] [v2:127.0.0.1:6802/986443870,v1:127.0.0.1:6803/986443870] exists,up 02f92415-04dc-48ea-a1f6-6d4335c1ad7a 2026-02-17T18:43:21.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:21.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:21.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:34: run: run_osd td/rados-striper 1 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/1 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:21.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/1' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/1/journal' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.54356/$cluster-$name.asok' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' 2026-02-17T18:43:21.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/1 2026-02-17T18:43:21.377 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:21.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1f9fc436-c43a-4daf-a7f1-46939cfb95fc 2026-02-17T18:43:21.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1f9fc436-c43a-4daf-a7f1-46939cfb95fc' 2026-02-17T18:43:21.378 INFO:tasks.workunit.client.0.trial068.stdout:add osd1 1f9fc436-c43a-4daf-a7f1-46939cfb95fc 2026-02-17T18:43:21.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:21.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDJtpRpWlHJFhAAmSiasoE5Z/fJkW8JhOZnZw== 2026-02-17T18:43:21.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDJtpRpWlHJFhAAmSiasoE5Z/fJkW8JhOZnZw=="}' 2026-02-17T18:43:21.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1f9fc436-c43a-4daf-a7f1-46939cfb95fc -i td/rados-striper/1/new.json 2026-02-17T18:43:21.482 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:21.488 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/1/new.json 2026-02-17T18:43:21.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJtpRpWlHJFhAAmSiasoE5Z/fJkW8JhOZnZw== --osd-uuid 1f9fc436-c43a-4daf-a7f1-46939cfb95fc 2026-02-17T18:43:21.503 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.502+0000 7fe6d03fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:21.505 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.503+0000 7fe6d03fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:21.505 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.504+0000 7fe6d03fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:21.506 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.505+0000 7fe6d03fd780 -1 bdev(0x2c54c00 td/rados-striper/1/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:21.506 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.505+0000 7fe6d03fd780 -1 bluestore(td/rados-striper/1) _read_fsid unparsable uuid 2026-02-17T18:43:21.832 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/1/keyring 2026-02-17T18:43:21.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:21.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T18:43:21.833 INFO:tasks.workunit.client.0.trial068.stdout:adding osd1 key to auth repository 2026-02-17T18:43:21.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:21.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T18:43:21.932 INFO:tasks.workunit.client.0.trial068.stdout:start osd.1 2026-02-17T18:43:21.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:21.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:21.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:21.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:21.945 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.944+0000 7f11cdc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:21.947 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.946+0000 7f11cdc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:21.948 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:21.947+0000 7f11cdc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:22.026 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T18:43:22.026 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:22.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T18:43:22.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:22.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:22.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:22.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:22.027 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:22.028 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:22.028 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:43:22.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:22.135 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:22.133+0000 7f11cdc46780 -1 Falling back to public interface 2026-02-17T18:43:22.404 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:22.403+0000 7f11cdc46780 -1 osd.1 0 log_to_monitors true 2026-02-17T18:43:23.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:23.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:23.119 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:23.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:23.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:23.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:43:23.214 INFO:tasks.workunit.client.0.trial068.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/2768537469,v1:127.0.0.1:6809/2768537469] [v2:127.0.0.1:6810/2768537469,v1:127.0.0.1:6811/2768537469] exists,up 1f9fc436-c43a-4daf-a7f1-46939cfb95fc 2026-02-17T18:43:23.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:23.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:23.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:23.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:35: run: run_osd td/rados-striper 2 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/2 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/2' 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/2/journal' 2026-02-17T18:43:23.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.54356/$cluster-$name.asok' 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:23.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/2 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:23.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e310e064-3d28-4493-a994-f95c7e4ebea8 2026-02-17T18:43:23.222 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 e310e064-3d28-4493-a994-f95c7e4ebea8' 2026-02-17T18:43:23.222 INFO:tasks.workunit.client.0.trial068.stdout:add osd2 e310e064-3d28-4493-a994-f95c7e4ebea8 2026-02-17T18:43:23.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:23.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDLtpRpm2x4DRAA8fINyWWQdE/llk0UITBFSA== 2026-02-17T18:43:23.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDLtpRpm2x4DRAA8fINyWWQdE/llk0UITBFSA=="}' 2026-02-17T18:43:23.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e310e064-3d28-4493-a994-f95c7e4ebea8 -i td/rados-striper/2/new.json 2026-02-17T18:43:23.329 INFO:tasks.workunit.client.0.trial068.stdout:2 2026-02-17T18:43:23.334 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/2/new.json 2026-02-17T18:43:23.335 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDLtpRpm2x4DRAA8fINyWWQdE/llk0UITBFSA== --osd-uuid e310e064-3d28-4493-a994-f95c7e4ebea8 2026-02-17T18:43:23.349 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.348+0000 7fcc45a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:23.350 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.349+0000 7fcc45a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:23.351 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.350+0000 7fcc45a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:23.352 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.351+0000 7fcc45a1a780 -1 bdev(0x2c2ec00 td/rados-striper/2/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:23.352 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.351+0000 7fcc45a1a780 -1 bluestore(td/rados-striper/2) _read_fsid unparsable uuid 2026-02-17T18:43:23.688 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/2/keyring 2026-02-17T18:43:23.688 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:23.689 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-17T18:43:23.689 INFO:tasks.workunit.client.0.trial068.stdout:adding osd2 key to auth repository 2026-02-17T18:43:23.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:23.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-17T18:43:23.788 INFO:tasks.workunit.client.0.trial068.stdout:start osd.2 2026-02-17T18:43:23.788 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=52713634-5167-4daa-afe8-eb8290ed75a2 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.54356/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:23.789 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:23.789 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:23.789 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:23.800 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.799+0000 7f2d11443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:23.802 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.801+0000 7f2d11443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:23.803 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.801+0000 7f2d11443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:23.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-17T18:43:23.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:23.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-17T18:43:23.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:23.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:23.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:23.882 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:23.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:23.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:23.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:43:23.972 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:23.971+0000 7f2d11443780 -1 Falling back to public interface 2026-02-17T18:43:23.977 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:24.259 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:24.258+0000 7f2d11443780 -1 osd.2 0 log_to_monitors true 2026-02-17T18:43:24.978 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:24.978 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:24.979 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:24.979 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:24.979 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:24.979 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-17T18:43:25.075 INFO:tasks.workunit.client.0.trial068.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/2702895524,v1:127.0.0.1:6817/2702895524] [v2:127.0.0.1:6818/2702895524,v1:127.0.0.1:6819/2702895524] exists,up e310e064-3d28-4493-a994-f95c7e4ebea8 2026-02-17T18:43:25.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:25.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:25.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:25.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:36: run: create_rbd_pool 2026-02-17T18:43:25.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:43:25.168 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:43:25.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:43:25.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:43:25.322 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:43:25.328 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:43:26.329 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:43:26.631 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:39: run: dd if=/dev/urandom of=td/rados-striper/toyfile bs=1234 count=1 2026-02-17T18:43:26.632 INFO:tasks.workunit.client.0.trial068.stderr:1+0 records in 2026-02-17T18:43:26.632 INFO:tasks.workunit.client.0.trial068.stderr:1+0 records out 2026-02-17T18:43:26.632 INFO:tasks.workunit.client.0.trial068.stderr:1234 bytes (1.2 kB, 1.2 KiB) copied, 3.7361e-05 s, 33.0 MB/s 2026-02-17T18:43:26.632 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:42: run: rados --pool rbd --striper put toyfile td/rados-striper/toyfile 2026-02-17T18:43:26.651 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: rados --pool rbd --striper stat toyfile 2026-02-17T18:43:26.651 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: cut -d , -f 2 2026-02-17T18:43:26.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-17T18:43:26.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: cut -d , -f 2 2026-02-17T18:43:26.680 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:47: run: echo ' size 1234' 2026-02-17T18:43:26.680 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:48: run: diff -w td/rados-striper/stripedStat td/rados-striper/refstat 2026-02-17T18:43:26.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:49: run: diff -w td/rados-striper/stat td/rados-striper/refstat 2026-02-17T18:43:26.682 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:50: run: rados --pool rbd stat toyfile 2026-02-17T18:43:26.696 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:51: run: grep -q 'No such file or directory' td/rados-striper/staterror 2026-02-17T18:43:26.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:54: run: rados --pool rbd --striper get toyfile td/rados-striper/stripedGroup 2026-02-17T18:43:26.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:55: run: diff -w td/rados-striper/toyfile td/rados-striper/stripedGroup 2026-02-17T18:43:26.714 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:56: run: rados --pool rbd get toyfile.0000000000000000 td/rados-striper/nonSTripedGroup 2026-02-17T18:43:26.730 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:57: run: diff -w td/rados-striper/toyfile td/rados-striper/nonSTripedGroup 2026-02-17T18:43:26.731 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:60: run: rados --pool rbd --striper truncate toyfile 12 2026-02-17T18:43:26.748 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: rados --pool rbd --striper stat toyfile 2026-02-17T18:43:26.748 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: cut -d , -f 2 2026-02-17T18:43:26.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-17T18:43:26.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: cut -d , -f 2 2026-02-17T18:43:26.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:63: run: echo ' size 12' 2026-02-17T18:43:26.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:64: run: diff -w td/rados-striper/stripedStat td/rados-striper/reftrunc 2026-02-17T18:43:26.778 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:65: run: diff -w td/rados-striper/stat td/rados-striper/reftrunc 2026-02-17T18:43:26.779 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:69: run: rados --pool rbd --striper setxattr toyfile somexattr somevalue 2026-02-17T18:43:26.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:70: run: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-17T18:43:26.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:71: run: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-17T18:43:26.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:72: run: echo somevalue 2026-02-17T18:43:26.825 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:73: run: diff -w td/rados-striper/xattrvalue td/rados-striper/refvalue 2026-02-17T18:43:26.826 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:74: run: diff -w td/rados-striper/xattrvalue2 td/rados-striper/refvalue 2026-02-17T18:43:26.826 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:75: run: rados --pool rbd --striper listxattr toyfile 2026-02-17T18:43:26.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:76: run: echo somexattr 2026-02-17T18:43:26.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:77: run: diff -w td/rados-striper/xattrlist td/rados-striper/reflist 2026-02-17T18:43:26.842 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: rados --pool rbd listxattr toyfile.0000000000000000 2026-02-17T18:43:26.842 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: grep -v striper 2026-02-17T18:43:26.855 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:79: run: diff -w td/rados-striper/xattrlist2 td/rados-striper/reflist 2026-02-17T18:43:26.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:80: run: rados --pool rbd --striper rmxattr toyfile somexattr 2026-02-17T18:43:26.871 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:82: run: local 'attr_not_found_str=No data available' 2026-02-17T18:43:26.872 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: uname 2026-02-17T18:43:26.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: '[' Linux = FreeBSD ']' 2026-02-17T18:43:26.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:85: run: expect_failure td/rados-striper 'No data available' rados --pool rbd --striper getxattr toyfile somexattr 2026-02-17T18:43:26.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T18:43:26.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T18:43:26.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-17T18:43:26.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T18:43:26.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T18:43:26.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-17T18:43:26.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T18:43:26.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T18:43:26.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-17T18:43:26.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T18:43:26.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:87: run: expect_failure td/rados-striper 'No data available' rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-17T18:43:26.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T18:43:26.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T18:43:26.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-17T18:43:26.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T18:43:26.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T18:43:26.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-17T18:43:26.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T18:43:26.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T18:43:26.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-17T18:43:26.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T18:43:26.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:91: run: rados --pool rbd --striper rm toyfile 2026-02-17T18:43:26.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:92: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd --striper stat toyfile 2026-02-17T18:43:26.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T18:43:26.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T18:43:26.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-17T18:43:26.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T18:43:26.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T18:43:26.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper stat toyfile 2026-02-17T18:43:26.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T18:43:26.934 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T18:43:26.934 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:94: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd stat toyfile.0000000000000000 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T18:43:26.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd stat toyfile.0000000000000000 2026-02-17T18:43:26.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T18:43:26.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T18:43:26.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-17T18:43:26.949 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T18:43:26.949 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:98: run: teardown td/rados-striper 2026-02-17T18:43:26.949 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-17T18:43:26.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:26.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-17T18:43:26.950 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:26.950 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:26.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:26.951 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:26.951 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:27.057 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:27.057 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:27.058 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:27.058 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:27.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:27.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:27.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:27.061 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:27.061 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:27.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:27.062 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:27.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:27.063 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:27.063 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-17T18:43:27.069 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:27.069 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.070 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:27.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.54356 2026-02-17T18:43:27.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:27.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:27.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-17T18:43:27.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/rados-striper 0 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:27.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:27.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:27.073 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:27.074 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:27.074 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:27.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:27.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:27.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:27.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:27.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:27.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:27.077 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:27.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:27.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T18:43:27.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-17T18:43:27.079 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:27.079 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.079 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.54356 2026-02-17T18:43:27.080 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.54356 2026-02-17T18:43:27.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:27.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:27.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-17T18:43:27.081 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T18:43:27.082 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T18:43:27.146 INFO:tasks.workunit:Running workunit misc/test-ceph-helpers.sh... 2026-02-17T18:43:27.146 DEBUG:teuthology.orchestra.run.trial068:workunit test misc/test-ceph-helpers.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh 2026-02-17T18:43:27.210 INFO:tasks.workunit.client.0.trial068.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T18:43:27.213 INFO:tasks.workunit.client.0.trial068.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T18:43:27.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: export PATH=./bin:.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:43:27.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: PATH=./bin:.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-17T18:43:27.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: export CEPH_MON=127.0.0.1:7109 2026-02-17T18:43:27.214 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: CEPH_MON=127.0.0.1:7109 2026-02-17T18:43:27.214 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2417: run_tests: export CEPH_ARGS 2026-02-17T18:43:27.214 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: uuidgen 2026-02-17T18:43:27.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: CEPH_ARGS+=' --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none ' 2026-02-17T18:43:27.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2419: run_tests: CEPH_ARGS+='--mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:27.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: export CEPH_CONF=/dev/null 2026-02-17T18:43:27.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: CEPH_CONF=/dev/null 2026-02-17T18:43:27.215 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: set 2026-02-17T18:43:27.215 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: sed -n -e 's/^\(test_[0-9a-z_]*\) .*/\1/p' 2026-02-17T18:43:27.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: local 'funcs=test_activate_osd 2026-02-17T18:43:27.217 INFO:tasks.workunit.client.0.trial068.stderr:test_activate_osd_after_mark_down 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_activate_osd_skip_benchmark 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_destroy_osd 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_display_logs 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_erasure_code_plugin_exists 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_expect_failure 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_file_with_random_data 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_flush_pg_stats 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_get_config 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_get_is_making_recovery_progress 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_get_last_scrub_stamp 2026-02-17T18:43:27.218 INFO:tasks.workunit.client.0.trial068.stderr:test_get_not_primary 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_num_active_clean 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_num_active_or_peered 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_num_pgs 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_op_scheduler 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_osd_id_used_by_pgs 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_osds 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_pg 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_primary 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_get_timeout_delays 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_is_clean 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_kill_daemon 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_kill_daemons 2026-02-17T18:43:27.219 INFO:tasks.workunit.client.0.trial068.stderr:test_objectstore_tool 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_pg_schedule_scrub 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_pg_scrub 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_repair 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_run_in_background 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_run_mon 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_run_osd 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_set_config 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_setup 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_teardown 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_wait_background 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_wait_for_clean 2026-02-17T18:43:27.220 INFO:tasks.workunit.client.0.trial068.stderr:test_wait_for_health_ok 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:test_wait_for_osd 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:test_wait_for_peered 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:test_wait_for_scrub 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:test_wait_osd_id_used_by_pgs' 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: local dir=td/ceph-helpers 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd td/ceph-helpers 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: test_activate_osd: local dir=td/ceph-helpers 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: test_activate_osd: setup td/ceph-helpers 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:27.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:27.222 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:27.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:27.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:27.222 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:27.222 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:27.222 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:27.224 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:27.224 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:27.224 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:27.225 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:27.226 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:27.226 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.226 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:27.226 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:27.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:27.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:27.227 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:27.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:27.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.227 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:27.228 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:27.228 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:27.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-17T18:43:27.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-17T18:43:27.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:27.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:27.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:881: test_activate_osd: run_mon td/ceph-helpers a 2026-02-17T18:43:27.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:27.229 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:27.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:27.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:27.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:27.230 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:43:27.246 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:27.246 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:27.246 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:27.246 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:27.246 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.246 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:27.246 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:27.247 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:27.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.273 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:27.274 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:27.274 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:27.274 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:27.317 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.317 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:27.317 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:27.317 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:27.317 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:27.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: test_activate_osd: run_mgr td/ceph-helpers x 2026-02-17T18:43:27.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:27.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:27.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:27.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:27.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:27.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:27.457 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:27.457 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:27.457 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:27.457 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:27.457 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.457 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:27.458 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:27.458 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:27.458 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:27.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:884: test_activate_osd: run_osd td/ceph-helpers 0 2026-02-17T18:43:27.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:43:27.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:27.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:27.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a4375909-d111-4abb-9656-f4d8598ca9eb 2026-02-17T18:43:27.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a4375909-d111-4abb-9656-f4d8598ca9eb' 2026-02-17T18:43:27.480 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 a4375909-d111-4abb-9656-f4d8598ca9eb 2026-02-17T18:43:27.480 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:27.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDPtpRp/YPuHBAASm+zjrFbsujFjkWIoVaOxw== 2026-02-17T18:43:27.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDPtpRp/YPuHBAASm+zjrFbsujFjkWIoVaOxw=="}' 2026-02-17T18:43:27.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a4375909-d111-4abb-9656-f4d8598ca9eb -i td/ceph-helpers/0/new.json 2026-02-17T18:43:27.589 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:27.596 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:43:27.597 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDPtpRp/YPuHBAASm+zjrFbsujFjkWIoVaOxw== --osd-uuid a4375909-d111-4abb-9656-f4d8598ca9eb 2026-02-17T18:43:27.613 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:27.611+0000 7f285d247780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:27.614 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:27.613+0000 7f285d247780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:27.615 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:27.614+0000 7f285d247780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:27.615 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:27.614+0000 7f285d247780 -1 bdev(0x3c5ac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:27.616 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:27.614+0000 7f285d247780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:43:28.026 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:43:28.026 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:28.026 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:28.027 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:28.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:28.130 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:28.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:28.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:28.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:28.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:28.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:28.144 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:28.143+0000 7fa48d619780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:28.146 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:28.145+0000 7fa48d619780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:28.146 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:28.145+0000 7fa48d619780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:28.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:28.234 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:28.234 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:28.234 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:28.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:28.395 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:28.393+0000 7fa48d619780 -1 Falling back to public interface 2026-02-17T18:43:28.698 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:28.697+0000 7fa48d619780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:29.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:29.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:29.333 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:29.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:29.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:29.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:29.431 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/887095160,v1:127.0.0.1:6801/887095160] [v2:127.0.0.1:6802/887095160,v1:127.0.0.1:6803/887095160] exists,up a4375909-d111-4abb-9656-f4d8598ca9eb 2026-02-17T18:43:29.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:29.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:29.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:29.433 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: get_asok_path osd.0 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: CEPH_ARGS= 2026-02-17T18:43:29.434 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:43:29.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: local 'backfills={"osd_max_backfills":"1"}' 2026-02-17T18:43:29.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: echo '{"osd_max_backfills":"1"}' 2026-02-17T18:43:29.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: grep --quiet osd_max_backfills 2026-02-17T18:43:29.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:889: test_activate_osd: kill_daemons td/ceph-helpers TERM osd 2026-02-17T18:43:29.481 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:29.481 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:29.481 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:29.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:29.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:29.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:891: test_activate_osd: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:29.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:29.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:29.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T18:43:29.794 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:29.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-17T18:43:29.795 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T18:43:29.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T18:43:29.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T18:43:29.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T18:43:29.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T18:43:29.806 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:29.805+0000 7fa475210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:29.807 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:29.806+0000 7fa475210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:29.808 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:29.807+0000 7fa475210780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:29.934 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:29.932+0000 7fa475210780 -1 Falling back to public interface 2026-02-17T18:43:30.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T18:43:30.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:30.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:30.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:30.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:30.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:30.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:30.021 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:30.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:30.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:30.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:30.254 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:30.253+0000 7fa475210780 -1 osd.0 5 log_to_monitors true 2026-02-17T18:43:31.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:31.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:31.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:31.250 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:31.250 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:31.250 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:31.477 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/4277860666,v1:127.0.0.1:6801/4277860666] [v2:127.0.0.1:6802/4277860666,v1:127.0.0.1:6803/4277860666] exists,up a4375909-d111-4abb-9656-f4d8598ca9eb 2026-02-17T18:43:31.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:31.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: get_asok_path osd.0 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: CEPH_ARGS= 2026-02-17T18:43:31.479 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:43:31.526 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: local 'backfills={"osd_max_backfills":"20"}' 2026-02-17T18:43:31.526 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:894: test_activate_osd: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:896: test_activate_osd: teardown td/ceph-helpers 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:31.527 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:31.632 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:31.633 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:31.633 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:31.634 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:31.635 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:31.635 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:31.635 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:31.636 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:31.636 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:31.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:31.637 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:31.638 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:31.638 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:31.639 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:31.642 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:31.642 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.642 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.642 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:31.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:31.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:31.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_after_mark_down td/ceph-helpers 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:900: test_activate_osd_after_mark_down: local dir=td/ceph-helpers 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:902: test_activate_osd_after_mark_down: setup td/ceph-helpers 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:31.645 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:31.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:31.646 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:31.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:31.647 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:31.647 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:31.647 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:31.647 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:31.647 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:31.648 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:31.648 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:31.649 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:31.649 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:31.649 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:31.650 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:31.650 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:31.651 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:31.651 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.651 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.651 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:31.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:31.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:31.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:31.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:31.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:31.654 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:904: test_activate_osd_after_mark_down: run_mon td/ceph-helpers a 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:31.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:31.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:31.698 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:31.700 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:31.701 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:31.701 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.701 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.701 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:31.702 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:31.702 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:31.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:31.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:31.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:31.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:31.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:31.744 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:31.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:31.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:905: test_activate_osd_after_mark_down: run_mgr td/ceph-helpers x 2026-02-17T18:43:31.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:31.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:31.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:31.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:31.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:31.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:31.884 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:31.884 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:31.884 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:31.885 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:31.885 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.885 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.885 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:31.885 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:31.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:31.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:907: test_activate_osd_after_mark_down: run_osd td/ceph-helpers 0 2026-02-17T18:43:31.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:43:31.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:31.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:31.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:31.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:31.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:31.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:31.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:31.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:31.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:31.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:31.906 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:31.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e28a8b9d-c4de-408b-b890-4ae239637ec4 2026-02-17T18:43:31.907 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 e28a8b9d-c4de-408b-b890-4ae239637ec4 2026-02-17T18:43:31.907 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e28a8b9d-c4de-408b-b890-4ae239637ec4' 2026-02-17T18:43:31.907 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:31.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDTtpRprbZmNhAA8l/ZkcodmgFr50FDhGgu8Q== 2026-02-17T18:43:31.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDTtpRprbZmNhAA8l/ZkcodmgFr50FDhGgu8Q=="}' 2026-02-17T18:43:31.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e28a8b9d-c4de-408b-b890-4ae239637ec4 -i td/ceph-helpers/0/new.json 2026-02-17T18:43:32.016 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:32.023 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:43:32.024 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDTtpRprbZmNhAA8l/ZkcodmgFr50FDhGgu8Q== --osd-uuid e28a8b9d-c4de-408b-b890-4ae239637ec4 2026-02-17T18:43:32.040 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.039+0000 7fd590631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:32.041 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.040+0000 7fd590631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:32.042 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.041+0000 7fd590631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:32.043 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.041+0000 7fd590631780 -1 bdev(0x2e6cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:32.043 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.041+0000 7fd590631780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:43:32.449 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:43:32.450 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:32.450 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:32.451 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:32.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:32.554 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:32.554 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:32.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:32.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:32.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:32.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:32.569 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.567+0000 7fa2e9435780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:32.570 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.569+0000 7fa2e9435780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:32.571 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.570+0000 7fa2e9435780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:32.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:32.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:32.655 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:32.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:32.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:32.751 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:32.749+0000 7fa2e9435780 -1 Falling back to public interface 2026-02-17T18:43:32.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:33.059 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:33.057+0000 7fa2e9435780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:33.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:33.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:33.753 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:33.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:33.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:33.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:33.851 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3920822085,v1:127.0.0.1:6801/3920822085] [v2:127.0.0.1:6802/3920822085,v1:127.0.0.1:6803/3920822085] exists,up e28a8b9d-c4de-408b-b890-4ae239637ec4 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:33.853 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:33.854 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:33.854 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-17T18:43:33.854 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:43:33.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"1"}' 2026-02-17T18:43:33.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: echo '{"osd_max_backfills":"1"}' 2026-02-17T18:43:33.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: grep --quiet osd_max_backfills 2026-02-17T18:43:33.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:912: test_activate_osd_after_mark_down: kill_daemons td/ceph-helpers TERM osd 2026-02-17T18:43:33.903 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:33.903 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:33.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:33.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:33.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:34.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:34.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:913: test_activate_osd_after_mark_down: ceph osd down 0 2026-02-17T18:43:34.232 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 is already down. 2026-02-17T18:43:34.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:914: test_activate_osd_after_mark_down: wait_for_osd down 0 2026-02-17T18:43:34.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-17T18:43:34.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:34.240 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:34.240 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:34.240 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:34.240 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:34.240 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:34.241 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:34.241 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-17T18:43:34.474 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3920822085,v1:127.0.0.1:6801/3920822085] [v2:127.0.0.1:6802/3920822085,v1:127.0.0.1:6803/3920822085] exists e28a8b9d-c4de-408b-b890-4ae239637ec4 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: test_activate_osd_after_mark_down: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:34.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:34.479 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T18:43:34.480 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-17T18:43:34.481 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:34.481 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T18:43:34.481 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:34.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-17T18:43:34.482 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T18:43:34.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T18:43:34.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T18:43:34.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T18:43:34.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T18:43:34.491 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:34.490+0000 7fc5ab438780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:34.493 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:34.491+0000 7fc5ab438780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:34.493 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:34.492+0000 7fc5ab438780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:34.614 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:34.613+0000 7fc5ab438780 -1 Falling back to public interface 2026-02-17T18:43:34.703 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T18:43:34.703 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:34.704 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:34.704 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:34.704 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:34.704 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:34.704 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:34.704 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:34.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:34.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:34.898 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:34.897+0000 7fc5ab438780 -1 osd.0 5 log_to_monitors true 2026-02-17T18:43:34.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:35.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:35.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:35.933 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:35.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:35.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:35.934 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:36.157 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/1786036829,v1:127.0.0.1:6801/1786036829] [v2:127.0.0.1:6802/1786036829,v1:127.0.0.1:6803/1786036829] exists,up e28a8b9d-c4de-408b-b890-4ae239637ec4 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.159 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.160 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:36.160 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-17T18:43:36.160 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_max_backfills 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"20"}' 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: test_activate_osd_after_mark_down: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: test_activate_osd_after_mark_down: teardown td/ceph-helpers 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:36.207 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:36.208 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:36.208 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:36.208 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:36.314 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:36.314 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:36.315 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:36.315 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:36.316 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:36.316 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:36.317 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:36.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:36.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:36.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:36.318 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:36.319 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:36.320 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:36.320 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:36.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:36.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.323 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:36.325 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:36.325 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_skip_benchmark td/ceph-helpers 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: test_activate_osd_skip_benchmark: local dir=td/ceph-helpers 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:927: test_activate_osd_skip_benchmark: setup td/ceph-helpers 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:36.326 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:36.327 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:36.327 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:36.327 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:36.327 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:36.327 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:36.328 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:36.328 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:36.328 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:36.329 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:36.329 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:36.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:36.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:36.330 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:36.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:36.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:36.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:36.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:36.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:36.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:36.334 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:36.334 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:36.334 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:36.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:36.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.335 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:36.336 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:36.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:36.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: test_activate_osd_skip_benchmark: run_mon td/ceph-helpers a 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:36.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:43:36.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:36.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:36.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:36.355 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:36.356 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.356 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.356 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:36.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:36.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:36.382 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.383 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.383 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:36.383 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:36.383 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:36.424 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:36.425 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:36.425 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.425 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.425 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:36.425 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:36.425 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:36.465 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:930: test_activate_osd_skip_benchmark: run_mgr td/ceph-helpers x 2026-02-17T18:43:36.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:36.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:36.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:36.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:36.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:36.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:36.565 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:36.566 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:36.566 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:36.566 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:36.566 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.566 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.566 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:36.567 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:36.567 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:36.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:933: test_activate_osd_skip_benchmark: run_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T18:43:36.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:43:36.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:36.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:36.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:36.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:36.586 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:36.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ac47909c-d7a2-4d9c-98cd-4d24644a5003 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ac47909c-d7a2-4d9c-98cd-4d24644a5003' 2026-02-17T18:43:36.588 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 ac47909c-d7a2-4d9c-98cd-4d24644a5003 2026-02-17T18:43:36.589 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:36.595 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDYtpRpY3ZoIxAA+R0tpbcMzmYznd5zcJ+X2w== 2026-02-17T18:43:36.595 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDYtpRpY3ZoIxAA+R0tpbcMzmYznd5zcJ+X2w=="}' 2026-02-17T18:43:36.595 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ac47909c-d7a2-4d9c-98cd-4d24644a5003 -i td/ceph-helpers/0/new.json 2026-02-17T18:43:36.698 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:36.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:43:36.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true --mkfs --key AQDYtpRpY3ZoIxAA+R0tpbcMzmYznd5zcJ+X2w== --osd-uuid ac47909c-d7a2-4d9c-98cd-4d24644a5003 2026-02-17T18:43:36.723 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:36.722+0000 7fc17cde3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:36.725 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:36.723+0000 7fc17cde3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:36.725 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:36.724+0000 7fc17cde3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:36.726 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:36.725+0000 7fc17cde3780 -1 bdev(0x3536c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:36.726 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:36.725+0000 7fc17cde3780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:43:37.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:43:37.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:37.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:37.114 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:37.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:37.217 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:37.218 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:37.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T18:43:37.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:37.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:37.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:37.233 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:37.232+0000 7fc8f9e69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:37.234 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:37.233+0000 7fc8f9e69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:37.235 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:37.234+0000 7fc8f9e69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:37.317 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:37.317 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:37.317 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:37.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:37.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:37.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:37.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:37.318 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:37.319 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:37.319 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:37.413 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:37.479 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:37.478+0000 7fc8f9e69780 -1 Falling back to public interface 2026-02-17T18:43:37.974 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:37.972+0000 7fc8f9e69780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:38.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:38.414 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:38.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:38.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:38.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:38.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:38.510 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1437186774,v1:127.0.0.1:6801/1437186774] [v2:127.0.0.1:6802/1437186774,v1:127.0.0.1:6803/1437186774] exists,up ac47909c-d7a2-4d9c-98cd-4d24644a5003 2026-02-17T18:43:38.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T18:43:38.512 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-17T18:43:38.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: local 'max_iops_hdd_def={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:38.561 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T18:43:38.562 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-17T18:43:38.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: local 'max_iops_ssd_def={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-17T18:43:38.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:940: test_activate_osd_skip_benchmark: kill_daemons td/ceph-helpers TERM osd 2026-02-17T18:43:38.607 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:38.607 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:38.608 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:38.608 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:38.608 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:38.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:38.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:941: test_activate_osd_skip_benchmark: ceph osd down 0 2026-02-17T18:43:39.142 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 is already down. 2026-02-17T18:43:39.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:942: test_activate_osd_skip_benchmark: wait_for_osd down 0 2026-02-17T18:43:39.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-17T18:43:39.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:39.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:39.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:39.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:39.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:39.151 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:39.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:39.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-17T18:43:39.378 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/1437186774,v1:127.0.0.1:6801/1437186774] [v2:127.0.0.1:6802/1437186774,v1:127.0.0.1:6803/1437186774] exists ac47909c-d7a2-4d9c-98cd-4d24644a5003 2026-02-17T18:43:39.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:39.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:39.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:39.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:946: test_activate_osd_skip_benchmark: activate_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T18:43:39.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:39.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T18:43:39.385 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:39.386 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-17T18:43:39.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T18:43:39.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T18:43:39.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T18:43:39.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T18:43:39.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T18:43:39.395 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:39.394+0000 7f6850e64780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:39.397 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:39.396+0000 7f6850e64780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:39.398 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:39.397+0000 7f6850e64780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:39.528 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:39.526+0000 7f6850e64780 -1 Falling back to public interface 2026-02-17T18:43:39.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T18:43:39.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:39.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:39.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:39.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:39.607 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:39.608 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:39.608 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:39.608 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:39.608 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:39.833 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:39.831+0000 7f6850e64780 -1 osd.0 5 log_to_monitors true 2026-02-17T18:43:39.835 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:40.836 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:40.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:40.837 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:40.838 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:40.838 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:40.838 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:41.061 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/643589436,v1:127.0.0.1:6801/643589436] [v2:127.0.0.1:6802/643589436,v1:127.0.0.1:6803/643589436] exists,up ac47909c-d7a2-4d9c-98cd-4d24644a5003 2026-02-17T18:43:41.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:41.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:41.062 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:41.062 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T18:43:41.062 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:41.062 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:41.062 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:41.063 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.063 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.063 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:41.063 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T18:43:41.063 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-17T18:43:41.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: local 'max_iops_hdd_after_boot={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-17T18:43:41.111 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-17T18:43:41.111 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:41.111 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:41.111 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:41.111 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.111 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.111 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:41.112 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-17T18:43:41.112 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-17T18:43:41.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: local 'max_iops_ssd_after_boot={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:953: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' = '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:954: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' = '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:956: test_activate_osd_skip_benchmark: teardown td/ceph-helpers 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:41.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:41.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:41.263 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:41.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:41.265 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:41.266 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:41.267 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:41.267 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:41.267 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:41.268 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:41.268 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:41.268 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:41.269 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:41.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:41.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:41.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:41.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.273 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:41.275 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:41.275 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_destroy_osd td/ceph-helpers 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:787: test_destroy_osd: local dir=td/ceph-helpers 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:789: test_destroy_osd: setup td/ceph-helpers 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:41.276 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:41.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:41.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:41.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:41.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:41.278 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:41.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:41.278 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:41.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:41.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:41.279 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:41.279 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:41.279 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:41.280 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:41.280 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:41.280 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:41.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:41.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:41.282 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:41.282 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.283 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:41.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:41.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:41.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:41.284 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:41.285 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.285 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.285 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:41.286 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:790: test_destroy_osd: run_mon td/ceph-helpers a 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:41.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:43:41.305 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:41.306 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:41.306 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:41.306 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:41.306 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.306 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.306 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:41.306 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:41.328 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:41.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:41.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:41.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:41.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:41.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:41.330 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:41.330 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:41.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:41.331 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:41.331 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.331 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.331 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:41.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:41.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:41.371 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:41.372 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:41.372 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:41.372 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:41.372 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:41.373 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:41.373 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:41.373 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:41.373 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:41.373 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.373 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.374 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:41.374 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:41.374 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:41.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:791: test_destroy_osd: run_mgr td/ceph-helpers x 2026-02-17T18:43:41.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:41.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:41.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:41.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:41.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:41.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:41.514 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:41.514 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:41.514 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:41.515 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:41.515 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.515 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.515 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:41.516 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:41.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:41.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:792: test_destroy_osd: run_osd td/ceph-helpers 0 2026-02-17T18:43:41.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:43:41.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:41.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:41.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:41.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:41.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:41.535 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:41.536 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=439a144e-77eb-4799-b693-23cfd0343736 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 439a144e-77eb-4799-b693-23cfd0343736' 2026-02-17T18:43:41.537 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 439a144e-77eb-4799-b693-23cfd0343736 2026-02-17T18:43:41.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:41.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDdtpRpOTleIBAAvNrTzk+6RuRIXHaI+s3new== 2026-02-17T18:43:41.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDdtpRpOTleIBAAvNrTzk+6RuRIXHaI+s3new=="}' 2026-02-17T18:43:41.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 439a144e-77eb-4799-b693-23cfd0343736 -i td/ceph-helpers/0/new.json 2026-02-17T18:43:41.646 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:41.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:43:41.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDdtpRpOTleIBAAvNrTzk+6RuRIXHaI+s3new== --osd-uuid 439a144e-77eb-4799-b693-23cfd0343736 2026-02-17T18:43:41.670 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:41.669+0000 7f7ee665a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:41.671 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:41.670+0000 7f7ee665a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:41.672 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:41.671+0000 7f7ee665a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:41.673 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:41.671+0000 7f7ee665a780 -1 bdev(0x2878c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:41.673 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:41.672+0000 7f7ee665a780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:43:42.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:43:42.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:42.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:42.130 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:42.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:42.234 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:42.235 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:42.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:42.236 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:42.236 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:42.236 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:42.249 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:42.247+0000 7f91c0031780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:42.250 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:42.249+0000 7f91c0031780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:42.251 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:42.250+0000 7f91c0031780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:42.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:42.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:42.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:42.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:42.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:42.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:42.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:42.337 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:42.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:42.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:42.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:42.495 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:42.494+0000 7f91c0031780 -1 Falling back to public interface 2026-02-17T18:43:42.848 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:42.846+0000 7f91c0031780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:43.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:43.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:43.436 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:43.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:43.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:43.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:43.533 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1381752740,v1:127.0.0.1:6801/1381752740] [v2:127.0.0.1:6802/1381752740,v1:127.0.0.1:6803/1381752740] exists,up 439a144e-77eb-4799-b693-23cfd0343736 2026-02-17T18:43:43.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:43.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:43.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:43.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:793: test_destroy_osd: destroy_osd td/ceph-helpers 0 2026-02-17T18:43:43.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/ceph-helpers 2026-02-17T18:43:43.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=0 2026-02-17T18:43:43.535 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.0 2026-02-17T18:43:43.724 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 is already out. 2026-02-17T18:43:43.730 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/ceph-helpers TERM osd.0 2026-02-17T18:43:43.731 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:43.731 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:43.732 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:43.732 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:43.732 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:44.037 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:44.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.0 2026-02-17T18:43:44.264 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 is already down. 2026-02-17T18:43:44.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.0 --yes-i-really-mean-it 2026-02-17T18:43:44.493 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 does not exist 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/ceph-helpers/0 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers/0 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers/0 KILL 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:44.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:44.502 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:44.503 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:44.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:44.503 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:44.504 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:44.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:44.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:44.505 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:44.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:44.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:44.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:44.507 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:44.508 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:44.508 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:44.508 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers/0 2026-02-17T18:43:44.510 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:44.510 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:44.510 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:44.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:44.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:44.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:44.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/ceph-helpers/0 2026-02-17T18:43:44.512 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: ceph osd dump 2026-02-17T18:43:44.512 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: grep 'osd. ' 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:795: test_destroy_osd: teardown td/ceph-helpers 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:44.738 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:44.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:44.845 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:44.845 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:44.846 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:44.847 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:44.847 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:44.847 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:44.848 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:44.848 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:44.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:44.849 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:44.850 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:44.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:44.851 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:44.853 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:44.853 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:44.854 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:44.854 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:44.855 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_display_logs td/ceph-helpers 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2158: test_display_logs: local dir=td/ceph-helpers 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2160: test_display_logs: setup td/ceph-helpers 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:44.856 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:44.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:44.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:44.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:44.857 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:44.857 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:44.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:44.858 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:44.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:44.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:44.859 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:44.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:44.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:44.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:44.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:44.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:44.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:44.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:44.863 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:44.863 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:44.863 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:44.863 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:44.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:44.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:44.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:44.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:44.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:44.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:44.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:44.866 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:44.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2161: test_display_logs: run_mon td/ceph-helpers a 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:44.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:43:44.886 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:44.886 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:44.887 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:44.887 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:44.887 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:44.887 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:44.887 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:44.887 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:44.910 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:44.912 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:44.913 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:44.914 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:44.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:44.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:44.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:44.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:44.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:44.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:44.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:44.955 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:44.956 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:44.996 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2162: test_display_logs: kill_daemons td/ceph-helpers 2026-02-17T18:43:44.997 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:44.997 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:44.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:44.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:44.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.102 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.102 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2163: test_display_logs: display_logs td/ceph-helpers 2026-02-17T18:43:45.103 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2148: display_logs: local dir=td/ceph-helpers 2026-02-17T18:43:45.103 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: display_logs: find td/ceph-helpers -maxdepth 1 -name '*.log' 2026-02-17T18:43:45.103 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-17T18:43:45.104 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2152: display_logs: echo '======================= td/ceph-helpers/mon.a.log' 2026-02-17T18:43:45.104 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2153: display_logs: cat td/ceph-helpers/mon.a.log 2026-02-17T18:43:45.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-17T18:43:45.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2164: test_display_logs: grep --quiet mon.a.log td/ceph-helpers/log.out 2026-02-17T18:43:45.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2165: test_display_logs: teardown td/ceph-helpers 2026-02-17T18:43:45.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.108 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.108 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.109 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.110 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.111 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.111 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.111 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.112 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.112 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.112 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.112 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.115 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.115 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.115 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.117 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_erasure_code_plugin_exists td/ceph-helpers 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: test_erasure_code_plugin_exists: local dir=td/ceph-helpers 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: test_erasure_code_plugin_exists: setup td/ceph-helpers 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.118 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.120 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.121 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.122 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.123 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.123 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.124 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.125 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.125 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.125 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:45.127 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:45.127 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.127 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.127 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:45.128 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:45.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:45.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:45.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:45.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: test_erasure_code_plugin_exists: run_mon td/ceph-helpers a 2026-02-17T18:43:45.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:45.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:45.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:45.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:45.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:45.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:43:45.147 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:45.147 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:45.147 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:45.147 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:45.148 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.148 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.148 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:45.148 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:45.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:45.174 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.175 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:45.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:45.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:45.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:45.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:45.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:45.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:45.217 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:45.217 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:45.218 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:45.218 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:45.219 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:45.219 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.219 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.219 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:45.219 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:45.219 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:45.262 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: test_erasure_code_plugin_exists: run_mgr td/ceph-helpers x 2026-02-17T18:43:45.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:45.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:45.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:45.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:45.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:45.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:45.364 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:45.364 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:45.364 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:45.364 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:45.364 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.365 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.365 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:45.366 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:45.366 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:45.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: test_erasure_code_plugin_exists: erasure_code_plugin_exists jerasure 2026-02-17T18:43:45.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=jerasure 2026-02-17T18:43:45.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-17T18:43:45.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-17T18:43:45.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-17T18:43:45.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-17T18:43:45.382 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-17T18:43:45.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='jerasure.*No such file' 2026-02-17T18:43:45.383 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=jerasure 2026-02-17T18:43:45.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-17T18:43:45.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-17T18:43:45.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-17T18:43:45.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-17T18:43:45.662 INFO:tasks.workunit.client.0.trial068.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-17T18:43:45.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-17T18:43:45.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: test_erasure_code_plugin_exists: erasure_code_plugin_exists FAKE 2026-02-17T18:43:45.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=FAKE 2026-02-17T18:43:45.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-17T18:43:45.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-17T18:43:45.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-17T18:43:45.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-17T18:43:45.671 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-17T18:43:45.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='FAKE.*No such file' 2026-02-17T18:43:45.671 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=FAKE 2026-02-17T18:43:45.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s='Error EIO: load dlopen(/usr/lib64/ceph/erasure-code/libec_FAKE.so): /usr/lib64/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory' 2026-02-17T18:43:45.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=5 2026-02-17T18:43:45.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 5 -eq 0 ']' 2026-02-17T18:43:45.755 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: echo Error EIO: load 'dlopen(/usr/lib64/ceph/erasure-code/libec_FAKE.so):' /usr/lib64/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory 2026-02-17T18:43:45.755 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: grep --quiet 'FAKE.*No such file' 2026-02-17T18:43:45.756 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2119: erasure_code_plugin_exists: status=1 2026-02-17T18:43:45.756 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 1 2026-02-17T18:43:45.756 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: test_erasure_code_plugin_exists: teardown td/ceph-helpers 2026-02-17T18:43:45.756 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.756 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.757 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.757 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.861 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.862 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.862 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.863 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.865 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.866 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.867 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.868 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_expect_failure td/ceph-helpers 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: test_expect_failure: local dir=td/ceph-helpers 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2039: test_expect_failure: setup td/ceph-helpers 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:45.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.874 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.874 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.875 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.875 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.876 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.876 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.876 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.877 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.878 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.878 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.879 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.879 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.880 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.880 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.880 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.880 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:45.881 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:45.881 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.882 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:45.882 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2040: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 1' 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T18:43:45.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 1' 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2042: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 0' 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T18:43:45.886 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T18:43:45.887 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 0' 2026-02-17T18:43:45.887 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2023: expect_failure: success=true 2026-02-17T18:43:45.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: true 2026-02-17T18:43:45.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-17T18:43:45.888 INFO:tasks.workunit.client.0.trial068.stderr:FAIL 2026-02-17T18:43:45.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-17T18:43:45.888 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2043: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T18:43:45.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo UNEXPECTED ; exit 1' 2026-02-17T18:43:45.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-17T18:43:45.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-17T18:43:45.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-17T18:43:45.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-17T18:43:45.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-17T18:43:45.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo UNEXPECTED ; exit 1' 2026-02-17T18:43:45.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-17T18:43:45.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-17T18:43:45.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T18:43:45.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-17T18:43:45.891 INFO:tasks.workunit.client.0.trial068.stderr:UNEXPECTED 2026-02-17T18:43:45.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-17T18:43:45.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: test_expect_failure: teardown td/ceph-helpers 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.895 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.896 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.896 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.897 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.897 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.897 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.898 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.898 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.898 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.899 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.899 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.900 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_file_with_random_data td/ceph-helpers 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2350: test_file_with_random_data: local dir=td/ceph-helpers 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2352: test_file_with_random_data: setup td/ceph-helpers 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.904 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.905 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.906 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.906 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.907 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.907 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.907 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.907 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.908 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.908 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.909 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.910 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.910 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.910 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.910 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.911 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.911 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.911 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:45.912 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:45.912 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.912 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.912 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:45.913 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:45.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:45.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:45.914 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:45.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: file_with_random_data 4000 2026-02-17T18:43:45.914 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2343: file_with_random_data: local size_bytes=4000 2026-02-17T18:43:45.914 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: mktemp 2026-02-17T18:43:45.915 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local file=/tmp/tmp.7SFKcBOBGC 2026-02-17T18:43:45.915 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.7SFKcBOBGC bs=4000 count=1 2026-02-17T18:43:45.916 INFO:tasks.workunit.client.0.trial068.stderr:1+0 records in 2026-02-17T18:43:45.916 INFO:tasks.workunit.client.0.trial068.stderr:1+0 records out 2026-02-17T18:43:45.916 INFO:tasks.workunit.client.0.trial068.stderr:4000 bytes (4.0 kB, 3.9 KiB) copied, 5.6366e-05 s, 71.0 MB/s 2026-02-17T18:43:45.916 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: printf %s /tmp/tmp.7SFKcBOBGC 2026-02-17T18:43:45.916 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: local file=/tmp/tmp.7SFKcBOBGC 2026-02-17T18:43:45.916 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2354: test_file_with_random_data: test -f /tmp/tmp.7SFKcBOBGC 2026-02-17T18:43:45.917 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: stat -c %s /tmp/tmp.7SFKcBOBGC 2026-02-17T18:43:45.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: test 4000 = 4000 2026-02-17T18:43:45.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2356: test_file_with_random_data: rm /tmp/tmp.7SFKcBOBGC 2026-02-17T18:43:45.918 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2357: test_file_with_random_data: teardown td/ceph-helpers 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.923 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.924 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.924 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.927 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.928 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.928 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.928 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_flush_pg_stats td/ceph-helpers 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2280: test_flush_pg_stats: local dir=td/ceph-helpers 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: test_flush_pg_stats: setup td/ceph-helpers 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:45.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:45.933 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:45.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:45.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:45.935 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:45.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:45.935 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:45.935 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:45.936 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:45.937 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:45.937 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:45.937 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:45.937 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.937 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.937 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:45.938 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:45.938 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:45.938 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:45.939 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:45.939 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.939 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:45.940 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2283: test_flush_pg_stats: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:45.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:45.960 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:45.960 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:45.960 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:45.960 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:45.960 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.960 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.960 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:45.961 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:45.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:45.983 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:45.984 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:45.984 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:45.984 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:45.985 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:45.986 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:45.986 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:46.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:46.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:46.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:46.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:46.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:46.029 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:46.029 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:46.029 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:46.029 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:46.030 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:46.030 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:46.030 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:46.030 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:46.030 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:46.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: test_flush_pg_stats: run_mgr td/ceph-helpers x 2026-02-17T18:43:46.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:46.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:46.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:46.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:46.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:46.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:46.172 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:46.172 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:46.172 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:46.172 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:46.172 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:46.173 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:46.173 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:46.173 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:46.173 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:46.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: test_flush_pg_stats: run_osd td/ceph-helpers 0 2026-02-17T18:43:46.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:43:46.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:46.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:46.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:46.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:46.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:46.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:46.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:46.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:46.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:46.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:46.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:46.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:46.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:46.194 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:46.195 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:46.195 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:46.195 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:46.195 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:46.195 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:46.196 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:46.196 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:46.196 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:46.196 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:46.196 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:46.196 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:46.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:46.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:46.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:46.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:46.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:46.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:46.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:46.198 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:46.198 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:46.199 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=58351164-2adf-4eee-9a57-1f12621e54c5 2026-02-17T18:43:46.199 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 58351164-2adf-4eee-9a57-1f12621e54c5 2026-02-17T18:43:46.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 58351164-2adf-4eee-9a57-1f12621e54c5' 2026-02-17T18:43:46.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:46.202 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDitpRpdsX6CxAAYknoSWcSCj9WfxijHmF15A== 2026-02-17T18:43:46.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDitpRpdsX6CxAAYknoSWcSCj9WfxijHmF15A=="}' 2026-02-17T18:43:46.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 58351164-2adf-4eee-9a57-1f12621e54c5 -i td/ceph-helpers/0/new.json 2026-02-17T18:43:46.304 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:46.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:43:46.311 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDitpRpdsX6CxAAYknoSWcSCj9WfxijHmF15A== --osd-uuid 58351164-2adf-4eee-9a57-1f12621e54c5 2026-02-17T18:43:46.328 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.326+0000 7f205cc3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:46.329 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.328+0000 7f205cc3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:46.330 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.329+0000 7f205cc3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:46.330 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.329+0000 7f205cc3a780 -1 bdev(0x2a8ec00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:46.330 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.329+0000 7f205cc3a780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:43:46.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:43:46.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:46.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:46.769 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:46.770 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:46.872 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:46.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:46.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:46.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:46.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:46.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:46.887 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.886+0000 7fb75e446780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:46.889 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.887+0000 7fb75e446780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:46.889 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:46.888+0000 7fb75e446780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:46.971 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:46.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:46.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:46.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:46.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:46.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:46.972 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:46.973 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:46.973 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:46.973 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:47.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:47.077 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:47.076+0000 7fb75e446780 -1 Falling back to public interface 2026-02-17T18:43:47.435 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:47.433+0000 7fb75e446780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:48.072 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:48.072 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:48.073 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:48.174 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/298422096,v1:127.0.0.1:6801/298422096] [v2:127.0.0.1:6802/298422096,v1:127.0.0.1:6803/298422096] exists,up 58351164-2adf-4eee-9a57-1f12621e54c5 2026-02-17T18:43:48.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:48.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:48.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:48.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: test_flush_pg_stats: create_rbd_pool 2026-02-17T18:43:48.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:43:48.270 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:43:48.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:43:48.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:43:48.595 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:43:48.602 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:43:49.604 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:43:49.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: test_flush_pg_stats: rados -p rbd put obj /etc/group 2026-02-17T18:43:49.899 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: test_flush_pg_stats: flush_pg_stats 2026-02-17T18:43:49.899 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:43:49.899 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:43:50.127 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:43:50.128 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:43:50.128 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:43:50.128 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:43:50.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:43:50.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:43:50.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:43:50.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:43:50.192 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:43:50.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:43:50.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:43:50.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:43:50.194 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:43:50.195 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:43:50.195 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:43:50.195 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:43:50.195 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:43:50.420 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:43:50.420 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: test_flush_pg_stats: local 'jq_filter=.pools | .[] | select(.name == "rbd") | .stats' 2026-02-17T18:43:50.421 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: ceph df detail --format=json 2026-02-17T18:43:50.421 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored' 2026-02-17T18:43:50.735 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: stored=8139 2026-02-17T18:43:50.735 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: ceph df detail --format=json 2026-02-17T18:43:50.736 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored_raw' 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: stored_raw=8139 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: test 8139 -gt 0 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2293: test_flush_pg_stats: test 8139 == 8139 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: test_flush_pg_stats: teardown td/ceph-helpers 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:51.066 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:51.067 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:51.067 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:51.067 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:51.067 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:51.172 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:51.173 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:51.173 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:51.174 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:51.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:51.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:51.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:51.176 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:51.177 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:51.177 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:51.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:51.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:51.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:51.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:51.182 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:51.182 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.183 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.183 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_config td/ceph-helpers 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: test_get_config: local dir=td/ceph-helpers 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: test_get_config: setup td/ceph-helpers 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:51.185 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:51.186 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:51.186 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:51.186 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:51.186 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:51.186 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:51.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:51.187 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:51.187 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:51.187 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:51.188 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:51.188 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:51.188 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:51.189 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:51.189 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:51.189 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:51.189 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:51.190 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:51.191 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:51.191 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:51.192 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:51.192 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.192 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:51.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:51.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:51.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:51.194 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:51.194 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.194 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:51.195 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1131: test_get_config: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:51.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:51.198 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:51.198 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:51.215 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:51.216 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:51.216 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:51.216 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:51.216 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.216 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.216 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:51.216 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:51.240 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:51.242 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:51.242 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:51.242 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:51.243 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.288 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.289 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:51.289 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:51.289 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: get_config mon a osd_pool_default_size 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_pool_default_size 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_pool_default_size 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:51.334 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:51.335 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:51.335 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.335 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.335 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:51.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:51.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get osd_pool_default_size 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: test 1 = 1 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1133: test_get_config: run_mgr td/ceph-helpers x 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:51.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:51.476 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:51.477 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:51.477 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:51.477 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:51.477 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.477 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.477 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:51.478 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:51.478 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:51.495 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: test_get_config: run_osd td/ceph-helpers 0 --osd_max_scrubs=3 2026-02-17T18:43:51.495 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:43:51.495 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:51.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:51.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_max_scrubs=3 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=afdcf3df-704d-47a9-95e3-71c9b1bff9b6 2026-02-17T18:43:51.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 afdcf3df-704d-47a9-95e3-71c9b1bff9b6' 2026-02-17T18:43:51.499 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 afdcf3df-704d-47a9-95e3-71c9b1bff9b6 2026-02-17T18:43:51.499 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:51.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDntpRpdYcZHhAAgnfa5tnMJNm7Zo0QK74q7A== 2026-02-17T18:43:51.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDntpRpdYcZHhAAgnfa5tnMJNm7Zo0QK74q7A=="}' 2026-02-17T18:43:51.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new afdcf3df-704d-47a9-95e3-71c9b1bff9b6 -i td/ceph-helpers/0/new.json 2026-02-17T18:43:51.609 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:51.615 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:43:51.616 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 --mkfs --key AQDntpRpdYcZHhAAgnfa5tnMJNm7Zo0QK74q7A== --osd-uuid afdcf3df-704d-47a9-95e3-71c9b1bff9b6 2026-02-17T18:43:51.632 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:51.631+0000 7fb9a2a13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:51.633 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:51.632+0000 7fb9a2a13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:51.634 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:51.633+0000 7fb9a2a13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:51.634 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:51.633+0000 7fb9a2a13780 -1 bdev(0x2da8c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:51.635 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:51.633+0000 7fb9a2a13780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:43:52.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:43:52.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:52.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:52.115 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:52.116 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:52.218 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:52.219 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:52.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 2026-02-17T18:43:52.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:52.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:52.220 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:52.234 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:52.233+0000 7f82f1820780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:52.236 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:52.234+0000 7f82f1820780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:52.236 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:52.235+0000 7f82f1820780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:52.317 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:52.317 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:52.317 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:52.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:52.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:52.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:52.318 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:52.318 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:52.319 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:52.319 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:52.415 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:52.449 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:52.447+0000 7f82f1820780 -1 Falling back to public interface 2026-02-17T18:43:52.813 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:52.811+0000 7f82f1820780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:53.417 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:53.417 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:53.418 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:53.418 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:53.418 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:53.419 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:53.517 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1318961494,v1:127.0.0.1:6801/1318961494] [v2:127.0.0.1:6802/1318961494,v1:127.0.0.1:6803/1318961494] exists,up afdcf3df-704d-47a9-95e3-71c9b1bff9b6 2026-02-17T18:43:53.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:53.519 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: get_config osd 0 osd_max_scrubs 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_max_scrubs 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_max_scrubs 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.520 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.521 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:43:53.521 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:53.521 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_max_scrubs 2026-02-17T18:43:53.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: test 3 = 3 2026-02-17T18:43:53.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1136: test_get_config: teardown td/ceph-helpers 2026-02-17T18:43:53.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:53.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:53.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:53.569 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:53.569 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:53.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:53.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:53.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:53.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:53.675 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:53.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:53.676 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:53.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:53.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:53.678 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:53.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:53.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:53.679 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:53.680 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:53.680 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:53.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:53.681 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:53.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:53.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.684 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:53.686 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:53.686 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_is_making_recovery_progress td/ceph-helpers 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: test_get_is_making_recovery_progress: local dir=td/ceph-helpers 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: test_get_is_making_recovery_progress: setup td/ceph-helpers 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:53.687 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:53.688 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:53.688 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:53.688 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:53.688 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:53.689 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:53.689 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:53.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:53.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:53.691 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:53.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:53.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:53.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:53.692 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:53.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:53.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:53.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:53.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:53.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.695 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.695 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:53.695 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:53.695 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:53.695 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:53.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:53.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:53.697 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:53.698 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1338: test_get_is_making_recovery_progress: run_mon td/ceph-helpers a 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:53.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:43:53.717 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:53.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:53.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:53.718 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:53.718 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.718 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:53.718 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:43:53.743 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:53.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:53.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:53.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:53.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:53.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:53.788 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:53.788 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:53.788 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:53.788 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:53.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:53.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: test_get_is_making_recovery_progress: run_mgr td/ceph-helpers x 2026-02-17T18:43:53.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:53.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:53.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:53.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:53.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:53.832 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:53.930 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:53.931 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:53.931 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:53.931 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:53.931 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:53.931 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:53.931 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:53.932 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:53.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:53.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: test_get_is_making_recovery_progress: get_is_making_recovery_progress 2026-02-17T18:43:53.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-17T18:43:53.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T18:43:53.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T18:43:53.948 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T18:43:53.948 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-17T18:43:53.948 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T18:43:54.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1341: test_get_is_making_recovery_progress: teardown td/ceph-helpers 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:54.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:54.051 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:54.051 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:54.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:54.156 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:54.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:54.157 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:54.158 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:54.158 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:54.159 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:54.160 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:54.161 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:54.161 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:54.161 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:54.162 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:54.162 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:54.164 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:54.164 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.164 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.164 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_last_scrub_stamp td/ceph-helpers 2026-02-17T18:43:54.166 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1524: test_get_last_scrub_stamp: local dir=td/ceph-helpers 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: test_get_last_scrub_stamp: setup td/ceph-helpers 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:54.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:54.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:54.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:54.168 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:54.169 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:54.169 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:54.169 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:54.169 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:54.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:54.171 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:54.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:54.172 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:54.172 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:54.174 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:54.174 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.174 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:54.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:54.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:54.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:54.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:54.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.176 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:54.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: test_get_last_scrub_stamp: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:54.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:54.196 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:54.196 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:54.196 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:54.196 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:54.196 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.196 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.196 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:54.197 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:43:54.221 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:54.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:54.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:54.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:54.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:54.222 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:54.223 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:54.223 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:54.223 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:54.223 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:54.224 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.224 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.224 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:54.224 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:54.224 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:54.267 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:54.268 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:54.269 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.269 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.269 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:54.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:54.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:54.309 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: test_get_last_scrub_stamp: run_mgr td/ceph-helpers x 2026-02-17T18:43:54.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:54.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:54.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:54.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:54.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:54.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:43:54.408 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:43:54.408 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:54.408 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:54.408 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:54.408 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.409 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.409 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:54.409 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:54.409 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:43:54.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: test_get_last_scrub_stamp: run_osd td/ceph-helpers 0 2026-02-17T18:43:54.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:43:54.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:43:54.428 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:43:54.429 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=efc028da-b804-4c35-b5da-5340ba1b6128 2026-02-17T18:43:54.430 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 efc028da-b804-4c35-b5da-5340ba1b6128 2026-02-17T18:43:54.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 efc028da-b804-4c35-b5da-5340ba1b6128' 2026-02-17T18:43:54.430 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:43:54.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDqtpRpQJHyGRAAFXnSLntyNnY+WpoD3JRwRQ== 2026-02-17T18:43:54.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDqtpRpQJHyGRAAFXnSLntyNnY+WpoD3JRwRQ=="}' 2026-02-17T18:43:54.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new efc028da-b804-4c35-b5da-5340ba1b6128 -i td/ceph-helpers/0/new.json 2026-02-17T18:43:54.536 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:54.542 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:43:54.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDqtpRpQJHyGRAAFXnSLntyNnY+WpoD3JRwRQ== --osd-uuid efc028da-b804-4c35-b5da-5340ba1b6128 2026-02-17T18:43:54.559 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:54.558+0000 7ff1f9fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:54.560 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:54.559+0000 7ff1f9fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:54.561 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:54.560+0000 7ff1f9fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:54.562 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:54.561+0000 7ff1f9fd8780 -1 bdev(0x3e22c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:43:54.562 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:54.561+0000 7ff1f9fd8780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:43:55.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:43:55.008 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:43:55.009 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:43:55.009 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:43:55.010 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:43:55.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:43:55.113 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:43:55.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:43:55.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:43:55.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:43:55.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:43:55.128 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:55.126+0000 7f5c65c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:55.129 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:55.128+0000 7f5c65c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:55.130 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:55.128+0000 7f5c65c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:43:55.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:43:55.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:43:55.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:43:55.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:43:55.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:43:55.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:55.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:43:55.213 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:43:55.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:55.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:55.308 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:43:55.320 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:55.319+0000 7f5c65c33780 -1 Falling back to public interface 2026-02-17T18:43:55.670 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:43:55.668+0000 7f5c65c33780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:43:56.309 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:43:56.309 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:43:56.310 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:43:56.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:43:56.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:43:56.310 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:43:56.407 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/509871336,v1:127.0.0.1:6801/509871336] [v2:127.0.0.1:6802/509871336,v1:127.0.0.1:6803/509871336] exists,up efc028da-b804-4c35-b5da-5340ba1b6128 2026-02-17T18:43:56.407 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:43:56.407 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:43:56.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:43:56.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1530: test_get_last_scrub_stamp: create_rbd_pool 2026-02-17T18:43:56.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:43:56.502 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:43:56.509 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:43:56.509 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:43:56.753 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:43:56.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:43:57.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:43:58.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1531: test_get_last_scrub_stamp: wait_for_clean 2026-02-17T18:43:58.041 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:43:58.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:43:58.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:43:58.042 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:43:58.043 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:43:58.043 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:43:58.043 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:43:58.043 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:43:58.043 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:43:58.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:43:58.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:43:58.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:43:58.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:43:58.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:43:58.122 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:43:58.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:43:58.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:43:58.349 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:43:58.349 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:43:58.413 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:43:58.413 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:43:58.413 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:43:58.413 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:43:58.414 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:43:58.414 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:43:58.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:43:58.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:43:58.415 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:43:58.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:43:58.416 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:43:58.416 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:43:58.416 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:43:58.642 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:43:58.642 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:43:58.643 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:43:58.643 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:43:58.954 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:43:58.954 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:43:58.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:43:58.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:43:58.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:43:58.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:43:58.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:43:58.955 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:43:59.176 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:43:59.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:43:59.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:43:59.177 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: get_last_scrub_stamp 1.0 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:43:59.491 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:43:59.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: stamp=2026-02-17T18:43:56.688549+0000 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1533: test_get_last_scrub_stamp: test -n 2026-02-17T18:43:56.688549+0000 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1534: test_get_last_scrub_stamp: teardown td/ceph-helpers 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:59.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:59.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:59.816 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:59.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:59.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:59.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:59.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:59.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:59.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:59.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:59.821 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:59.821 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:59.822 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:59.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:59.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:59.826 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:59.826 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:59.826 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:59.826 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:59.828 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:59.828 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_not_primary td/ceph-helpers 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: test_get_not_primary: local dir=td/ceph-helpers 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1231: test_get_not_primary: setup td/ceph-helpers 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:43:59.829 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:43:59.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:43:59.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:43:59.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:43:59.830 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:43:59.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:43:59.831 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:43:59.831 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:43:59.832 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:43:59.832 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:43:59.832 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:59.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:43:59.833 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:43:59.833 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:43:59.834 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:43:59.835 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:43:59.835 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:43:59.836 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:43:59.836 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:59.836 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:59.836 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:43:59.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:43:59.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:43:59.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:43:59.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:43:59.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:59.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:59.838 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:43:59.839 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:43:59.840 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: test_get_not_primary: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:43:59.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-17T18:43:59.859 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:43:59.859 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:43:59.859 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:43:59.859 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:43:59.859 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:59.859 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:59.860 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:43:59.860 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T18:43:59.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:43:59.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:43:59.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:59.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:59.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:43:59.883 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:43:59.883 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:59.883 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:59.884 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:59.884 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:59.885 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:59.885 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:59.885 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:59.885 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:59.885 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:43:59.930 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:43:59.930 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:43:59.930 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:43:59.930 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:43:59.930 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:43:59.930 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:43:59.931 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:43:59.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: test_get_not_primary: run_mgr td/ceph-helpers x 2026-02-17T18:43:59.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:43:59.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:43:59.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:43:59.973 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:43:59.973 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:43:59.973 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:00.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:00.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:00.075 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:00.075 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:00.076 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:00.076 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:00.076 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:00.076 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:00.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:00.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: test_get_not_primary: run_osd td/ceph-helpers 0 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:00.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:00.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:00.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=199df11b-9fa5-48ff-8313-9442550d60f2 2026-02-17T18:44:00.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 199df11b-9fa5-48ff-8313-9442550d60f2' 2026-02-17T18:44:00.099 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 199df11b-9fa5-48ff-8313-9442550d60f2 2026-02-17T18:44:00.099 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:00.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDwtpRpJQc2BhAA4fW/1Edy7oU0bxnfQbFNIw== 2026-02-17T18:44:00.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDwtpRpJQc2BhAA4fW/1Edy7oU0bxnfQbFNIw=="}' 2026-02-17T18:44:00.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 199df11b-9fa5-48ff-8313-9442550d60f2 -i td/ceph-helpers/0/new.json 2026-02-17T18:44:00.209 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:00.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:00.216 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDwtpRpJQc2BhAA4fW/1Edy7oU0bxnfQbFNIw== --osd-uuid 199df11b-9fa5-48ff-8313-9442550d60f2 2026-02-17T18:44:00.233 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.232+0000 7f6ebebfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:00.234 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.233+0000 7f6ebebfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:00.235 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.234+0000 7f6ebebfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:00.236 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.234+0000 7f6ebebfd780 -1 bdev(0x3682c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:00.236 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.234+0000 7f6ebebfd780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:44:00.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:44:00.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:00.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:44:00.668 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:44:00.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:00.773 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:44:00.773 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:44:00.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:00.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:00.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:00.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:00.788 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.787+0000 7f232d038780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:00.790 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.789+0000 7f232d038780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:00.791 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:00.789+0000 7f232d038780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:00.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:44:00.872 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:00.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:44:00.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:00.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:00.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:00.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:00.873 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:00.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:00.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:00.970 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:01.012 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:01.011+0000 7f232d038780 -1 Falling back to public interface 2026-02-17T18:44:01.436 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:01.435+0000 7f232d038780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:44:01.971 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:01.971 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:01.972 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:01.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:01.972 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:01.973 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:02.071 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3355435985,v1:127.0.0.1:6801/3355435985] [v2:127.0.0.1:6802/3355435985,v1:127.0.0.1:6803/3355435985] exists,up 199df11b-9fa5-48ff-8313-9442550d60f2 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1235: test_get_not_primary: run_osd td/ceph-helpers 1 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-17T18:44:02.075 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:02.076 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:02.077 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:02.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:02.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:02.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:02.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-17T18:44:02.078 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:02.078 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=319e25db-a7f0-4954-8ffd-1eaa63813557 2026-02-17T18:44:02.078 INFO:tasks.workunit.client.0.trial068.stdout:add osd1 319e25db-a7f0-4954-8ffd-1eaa63813557 2026-02-17T18:44:02.079 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 319e25db-a7f0-4954-8ffd-1eaa63813557' 2026-02-17T18:44:02.079 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:02.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDytpRpH2gFBRAAn9gMz6hd2MaukC8/r7BYpw== 2026-02-17T18:44:02.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDytpRpH2gFBRAAn9gMz6hd2MaukC8/r7BYpw=="}' 2026-02-17T18:44:02.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 319e25db-a7f0-4954-8ffd-1eaa63813557 -i td/ceph-helpers/1/new.json 2026-02-17T18:44:02.187 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:02.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-17T18:44:02.195 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDytpRpH2gFBRAAn9gMz6hd2MaukC8/r7BYpw== --osd-uuid 319e25db-a7f0-4954-8ffd-1eaa63813557 2026-02-17T18:44:02.211 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.210+0000 7fbfacc6b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:02.213 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.212+0000 7fbfacc6b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:02.213 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.212+0000 7fbfacc6b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:02.214 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.213+0000 7fbfacc6b780 -1 bdev(0x2ee0c00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:02.214 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.213+0000 7fbfacc6b780 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-17T18:44:02.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-17T18:44:02.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:02.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T18:44:02.544 INFO:tasks.workunit.client.0.trial068.stdout:adding osd1 key to auth repository 2026-02-17T18:44:02.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:02.865 INFO:tasks.workunit.client.0.trial068.stdout:start osd.1 2026-02-17T18:44:02.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T18:44:02.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:02.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:02.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:02.866 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:02.879 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.878+0000 7f59f1e65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:02.880 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.879+0000 7f59f1e65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:02.881 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:02.880+0000 7f59f1e65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:03.005 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:03.003+0000 7f59f1e65780 -1 Falling back to public interface 2026-02-17T18:44:03.092 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T18:44:03.093 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:03.093 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T18:44:03.093 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:03.093 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:03.093 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:03.093 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:03.094 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:03.094 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:03.094 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:44:03.306 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:03.304+0000 7f59f1e65780 -1 osd.1 0 log_to_monitors true 2026-02-17T18:44:03.321 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:04.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:04.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:04.323 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:04.323 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:04.323 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:04.324 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:44:04.548 INFO:tasks.workunit.client.0.trial068.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/491921350,v1:127.0.0.1:6811/491921350] [v2:127.0.0.1:6812/491921350,v1:127.0.0.1:6813/491921350] exists,up 319e25db-a7f0-4954-8ffd-1eaa63813557 2026-02-17T18:44:04.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:04.550 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:04.550 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:04.550 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1236: test_get_not_primary: create_rbd_pool 2026-02-17T18:44:04.550 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:44:04.772 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:44:04.779 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:44:04.779 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:44:05.025 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:44:05.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:44:06.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:44:06.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1237: test_get_not_primary: wait_for_clean 2026-02-17T18:44:06.314 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:44:06.314 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:44:06.314 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:44:06.314 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:44:06.315 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:44:06.315 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:44:06.315 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:44:06.315 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:44:06.315 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:44:06.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:44:06.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:44:06.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:44:06.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:44:06.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:44:06.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:44:06.624 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:44:06.625 INFO:tasks.workunit.client.0.trial068.stderr:1' 2026-02-17T18:44:06.625 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:44:06.625 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:06.625 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:44:06.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:44:06.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:44:06.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:44:06.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:06.692 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:44:06.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-17T18:44:06.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-17T18:44:06.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-17T18:44:06.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:06.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:06.759 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:06.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:44:06.760 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:06.760 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:06.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:44:06.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:44:06.761 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:44:06.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:06.987 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:06.987 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:07.988 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:44:07.988 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:08.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-17T18:44:08.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:08.212 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705667 2026-02-17T18:44:08.212 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:08.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:44:08.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-17T18:44:08.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:08.214 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705667 2026-02-17T18:44:08.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-17T18:44:08.215 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705667 2026-02-17T18:44:08.215 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:44:08.439 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-17T18:44:08.439 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:44:08.440 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:08.440 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:44:08.754 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:08.976 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:44:08.976 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:44:08.976 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:08.976 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:09.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:44:09.288 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:44:09.288 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:44:09.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: get_primary rbd GROUP 2026-02-17T18:44:09.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-17T18:44:09.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-17T18:44:09.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-17T18:44:09.288 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-17T18:44:09.515 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: local primary=1 2026-02-17T18:44:09.515 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: get_not_primary rbd GROUP 2026-02-17T18:44:09.515 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-17T18:44:09.516 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=GROUP 2026-02-17T18:44:09.516 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd GROUP 2026-02-17T18:44:09.516 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-17T18:44:09.516 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-17T18:44:09.516 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-17T18:44:09.516 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-17T18:44:09.744 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-17T18:44:09.744 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd GROUP 2026-02-17T18:44:09.744 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T18:44:09.974 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: local not_primary=0 2026-02-17T18:44:09.974 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1240: test_get_not_primary: test 0 '!=' 1 2026-02-17T18:44:09.974 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1241: test_get_not_primary: test 0 = 0 -o 0 = 1 2026-02-17T18:44:09.974 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1242: test_get_not_primary: teardown td/ceph-helpers 2026-02-17T18:44:09.974 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:09.975 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:09.975 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:09.975 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:09.975 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:09.975 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:09.975 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:09.975 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:10.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:10.081 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:10.082 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:10.083 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:10.084 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:10.084 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:10.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:10.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:10.085 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:10.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:10.086 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:10.086 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:10.087 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:10.087 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:10.092 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:10.092 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.093 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.093 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_clean td/ceph-helpers 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: test_get_num_active_clean: local dir=td/ceph-helpers 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: test_get_num_active_clean: setup td/ceph-helpers 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:44:10.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:10.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:10.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:10.096 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:10.096 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:10.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:10.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:10.096 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:10.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:10.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:10.097 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:10.097 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:10.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:10.098 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:10.098 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:10.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:10.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:10.099 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:10.100 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:10.100 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:10.101 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:10.102 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:10.102 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:10.103 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.103 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.103 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:10.103 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:10.104 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:10.104 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:44:10.104 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:44:10.104 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.105 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.105 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:44:10.105 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: test_get_num_active_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:44:10.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:10.125 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:44:10.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:10.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:10.126 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:10.126 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.126 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.126 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:10.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:10.148 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:44:10.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:44:10.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:10.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:10.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:10.151 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:44:10.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:44:10.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:10.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:10.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:44:10.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:44:10.193 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:10.194 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:10.194 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:10.195 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:10.195 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.195 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.195 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:10.195 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:10.195 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:44:10.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: test_get_num_active_clean: run_mgr td/ceph-helpers x 2026-02-17T18:44:10.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:44:10.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:44:10.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:44:10.236 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:44:10.236 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:44:10.236 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:10.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:10.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:10.335 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:10.335 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:10.336 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.336 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.336 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:10.336 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:10.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: test_get_num_active_clean: run_osd td/ceph-helpers 0 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:10.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:10.355 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:10.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:10.357 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:10.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=389fa010-e7e1-4feb-b6a4-014b4d2b8801 2026-02-17T18:44:10.357 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 389fa010-e7e1-4feb-b6a4-014b4d2b8801' 2026-02-17T18:44:10.358 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 389fa010-e7e1-4feb-b6a4-014b4d2b8801 2026-02-17T18:44:10.358 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:10.365 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD6tpRpR8GwFRAAVZQX8WL+MAdlw/7kKLsfiw== 2026-02-17T18:44:10.365 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD6tpRpR8GwFRAAVZQX8WL+MAdlw/7kKLsfiw=="}' 2026-02-17T18:44:10.365 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 389fa010-e7e1-4feb-b6a4-014b4d2b8801 -i td/ceph-helpers/0/new.json 2026-02-17T18:44:10.469 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:10.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:10.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD6tpRpR8GwFRAAVZQX8WL+MAdlw/7kKLsfiw== --osd-uuid 389fa010-e7e1-4feb-b6a4-014b4d2b8801 2026-02-17T18:44:10.493 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:10.492+0000 7f4456e5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:10.495 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:10.493+0000 7f4456e5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:10.496 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:10.494+0000 7f4456e5a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:10.496 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:10.495+0000 7f4456e5a780 -1 bdev(0x3654c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:10.496 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:10.495+0000 7f4456e5a780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:44:10.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:44:10.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:10.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:44:10.940 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:44:10.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:11.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:44:11.042 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:44:11.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:11.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:11.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:11.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:11.057 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:11.055+0000 7f892f610780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:11.058 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:11.057+0000 7f892f610780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:11.059 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:11.058+0000 7f892f610780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:11.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:44:11.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:11.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:44:11.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:11.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:11.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:11.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:11.140 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:11.140 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:11.140 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:11.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:11.242 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:11.241+0000 7f892f610780 -1 Falling back to public interface 2026-02-17T18:44:11.690 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:11.689+0000 7f892f610780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:44:12.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:12.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:12.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:12.237 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:12.238 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:12.238 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:12.336 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1370084846,v1:127.0.0.1:6801/1370084846] [v2:127.0.0.1:6802/1370084846,v1:127.0.0.1:6803/1370084846] exists,up 389fa010-e7e1-4feb-b6a4-014b4d2b8801 2026-02-17T18:44:12.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:12.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:12.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:12.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1369: test_get_num_active_clean: create_rbd_pool 2026-02-17T18:44:12.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:44:12.432 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:44:12.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:44:12.438 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:44:12.711 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:44:12.719 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:44:13.720 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:44:13.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1370: test_get_num_active_clean: wait_for_clean 2026-02-17T18:44:13.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:44:13.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:44:13.998 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:44:13.999 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:44:13.999 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:44:14.000 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:44:14.000 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:44:14.000 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:44:14.000 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:44:14.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:44:14.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:44:14.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:44:14.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:44:14.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:44:14.081 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:44:14.309 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:44:14.309 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:44:14.309 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:14.310 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:44:14.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:44:14.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:44:14.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:44:14.376 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:14.376 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:14.377 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:14.377 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:44:14.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:14.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:14.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:44:14.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:44:14.379 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:44:14.379 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:14.609 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:14.609 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:15.610 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:44:15.610 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:15.836 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:15.836 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:16.837 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:44:16.837 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:17.067 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:44:17.068 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:44:17.068 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:17.068 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:17.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:44:17.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:44:17.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:44:17.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:44:17.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:44:17.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:44:17.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:44:17.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:17.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:44:17.608 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:44:17.608 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:17.608 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:17.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: get_num_active_clean 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:44:17.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:18.141 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: local num_active_clean=4 2026-02-17T18:44:18.141 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1372: test_get_num_active_clean: test 4 = 4 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1373: test_get_num_active_clean: teardown td/ceph-helpers 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:18.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:18.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:18.249 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:18.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:18.250 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:18.252 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:18.252 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:18.252 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:18.254 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:18.254 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:18.254 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:18.255 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:18.255 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:18.255 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:18.256 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:18.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:18.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.259 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.259 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:18.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:18.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:18.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:44:18.263 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_or_peered td/ceph-helpers 2026-02-17T18:44:18.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1393: test_get_num_active_or_peered: local dir=td/ceph-helpers 2026-02-17T18:44:18.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1395: test_get_num_active_or_peered: setup td/ceph-helpers 2026-02-17T18:44:18.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:44:18.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:44:18.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:18.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:18.264 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:18.265 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:18.265 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:18.265 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:18.265 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:18.265 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:18.265 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:18.265 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:18.266 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:18.266 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:18.269 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:18.269 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:18.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:18.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:18.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:18.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:18.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:18.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:18.270 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:18.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:18.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:18.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:18.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:18.271 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:18.272 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:44:18.274 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:44:18.275 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.275 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.275 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:44:18.275 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:44:18.275 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:44:18.275 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:44:18.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:44:18.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1396: test_get_num_active_or_peered: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:18.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:44:18.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:44:18.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:44:18.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:44:18.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:44:18.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:18.290 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:44:18.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:18.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:18.291 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:18.291 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.291 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:18.291 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:18.314 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:44:18.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:44:18.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:18.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:18.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:44:18.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:44:18.316 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:18.317 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:44:18.361 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:44:18.362 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:18.362 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:18.362 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:44:18.362 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:44:18.362 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:18.362 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:18.362 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:18.363 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:18.363 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.363 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.363 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:18.363 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:18.363 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:44:18.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1397: test_get_num_active_or_peered: run_mgr td/ceph-helpers x 2026-02-17T18:44:18.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:44:18.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:44:18.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:44:18.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:44:18.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:44:18.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:18.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:18.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:18.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:18.502 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:18.502 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.503 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:18.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:18.503 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:18.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1398: test_get_num_active_or_peered: run_osd td/ceph-helpers 0 2026-02-17T18:44:18.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:18.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:18.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:18.524 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d3ecc492-9eb4-45c0-80c4-daeee32493f8 2026-02-17T18:44:18.525 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d3ecc492-9eb4-45c0-80c4-daeee32493f8' 2026-02-17T18:44:18.526 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 d3ecc492-9eb4-45c0-80c4-daeee32493f8 2026-02-17T18:44:18.526 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:18.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQACt5RpAUG5HxAAPEJuvr9wbvNum/IL9zTNOQ== 2026-02-17T18:44:18.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQACt5RpAUG5HxAAPEJuvr9wbvNum/IL9zTNOQ=="}' 2026-02-17T18:44:18.534 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d3ecc492-9eb4-45c0-80c4-daeee32493f8 -i td/ceph-helpers/0/new.json 2026-02-17T18:44:18.636 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:18.643 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:18.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQACt5RpAUG5HxAAPEJuvr9wbvNum/IL9zTNOQ== --osd-uuid d3ecc492-9eb4-45c0-80c4-daeee32493f8 2026-02-17T18:44:18.660 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:18.658+0000 7f6762a44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:18.661 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:18.660+0000 7f6762a44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:18.662 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:18.661+0000 7f6762a44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:18.662 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:18.661+0000 7f6762a44780 -1 bdev(0x368ec00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:18.663 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:18.661+0000 7f6762a44780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:44:19.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:44:19.069 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:19.070 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:44:19.070 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:44:19.071 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:19.173 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:44:19.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:44:19.174 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:19.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:19.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:19.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:19.187 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:19.186+0000 7f12c6a0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:19.189 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:19.188+0000 7f12c6a0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:19.190 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:19.188+0000 7f12c6a0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:19.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:44:19.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:19.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:44:19.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:19.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:19.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:19.274 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:19.275 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:19.275 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:19.276 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:19.372 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:19.412 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:19.411+0000 7f12c6a0b780 -1 Falling back to public interface 2026-02-17T18:44:19.734 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:19.733+0000 7f12c6a0b780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:44:20.373 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:20.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:20.374 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:20.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:20.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:20.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:20.471 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3669646326,v1:127.0.0.1:6801/3669646326] [v2:127.0.0.1:6802/3669646326,v1:127.0.0.1:6803/3669646326] exists,up d3ecc492-9eb4-45c0-80c4-daeee32493f8 2026-02-17T18:44:20.472 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:20.472 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:20.472 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:20.472 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1399: test_get_num_active_or_peered: create_rbd_pool 2026-02-17T18:44:20.473 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:44:20.564 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:44:20.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:44:20.570 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:44:20.846 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:44:20.853 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:44:21.854 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:44:22.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1400: test_get_num_active_or_peered: wait_for_clean 2026-02-17T18:44:22.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:44:22.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:44:22.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:44:22.134 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:44:22.135 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:44:22.135 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:44:22.135 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:44:22.135 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:44:22.136 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:44:22.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:44:22.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:44:22.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:44:22.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:44:22.218 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:44:22.218 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:44:22.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:44:22.442 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:44:22.442 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:22.442 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:44:22.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:44:22.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:44:22.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:44:22.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:22.506 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:22.506 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:22.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:44:22.508 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:22.508 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:22.508 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:44:22.509 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:44:22.509 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:44:22.509 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:22.733 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:22.734 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:23.734 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:44:23.735 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:23.960 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:23.960 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:24.961 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:44:24.962 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:25.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:44:25.200 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:44:25.201 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:25.201 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:25.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:44:25.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:44:25.516 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:44:25.517 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:44:25.517 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:44:25.517 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:44:25.517 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:44:25.517 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:25.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:44:25.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:44:25.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:25.744 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:26.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:44:26.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:44:26.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:44:26.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: get_num_active_or_peered 2026-02-17T18:44:26.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-17T18:44:26.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-17T18:44:26.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-17T18:44:26.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-17T18:44:26.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:26.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: local num_peered=4 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1402: test_get_num_active_or_peered: test 4 = 4 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1403: test_get_num_active_or_peered: teardown td/ceph-helpers 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:26.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:26.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:26.388 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:26.389 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:26.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:26.390 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:26.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:26.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:26.392 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:26.393 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:26.393 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:26.393 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:26.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:26.394 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:26.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:26.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:26.399 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:26.399 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.399 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.400 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:26.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:26.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:26.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:44:26.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_pgs td/ceph-helpers 2026-02-17T18:44:26.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1420: test_get_num_pgs: local dir=td/ceph-helpers 2026-02-17T18:44:26.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1422: test_get_num_pgs: setup td/ceph-helpers 2026-02-17T18:44:26.402 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:26.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:26.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:26.405 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:26.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:26.405 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:26.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:26.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:26.405 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:26.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:26.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:26.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:26.407 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:26.407 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:26.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:26.408 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:26.409 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:26.409 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.409 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.409 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:26.410 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:26.410 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:26.410 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:44:26.411 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:44:26.412 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.412 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.412 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1423: test_get_num_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:44:26.414 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:26.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:26.457 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:44:26.459 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:44:26.459 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:26.459 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:26.459 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:44:26.459 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:26.460 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:26.504 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.505 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.505 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:26.505 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:26.505 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:44:26.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1424: test_get_num_pgs: run_mgr td/ceph-helpers x 2026-02-17T18:44:26.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:44:26.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:44:26.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:44:26.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:44:26.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:44:26.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:26.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:26.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:26.646 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:26.647 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:26.648 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.648 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.648 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:26.648 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:26.648 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:26.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: test_get_num_pgs: run_osd td/ceph-helpers 0 2026-02-17T18:44:26.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:26.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:26.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:26.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:26.669 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:26.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=328c154a-2ace-4276-806a-34df25ba08cb 2026-02-17T18:44:26.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 328c154a-2ace-4276-806a-34df25ba08cb' 2026-02-17T18:44:26.670 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 328c154a-2ace-4276-806a-34df25ba08cb 2026-02-17T18:44:26.671 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:26.676 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAKt5RpfJRHKBAAH/y/AqywYmJUjNhcZoPZiQ== 2026-02-17T18:44:26.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAKt5RpfJRHKBAAH/y/AqywYmJUjNhcZoPZiQ=="}' 2026-02-17T18:44:26.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 328c154a-2ace-4276-806a-34df25ba08cb -i td/ceph-helpers/0/new.json 2026-02-17T18:44:26.778 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:26.784 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:26.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAKt5RpfJRHKBAAH/y/AqywYmJUjNhcZoPZiQ== --osd-uuid 328c154a-2ace-4276-806a-34df25ba08cb 2026-02-17T18:44:26.801 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:26.799+0000 7f4a37819780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:26.802 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:26.801+0000 7f4a37819780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:26.803 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:26.802+0000 7f4a37819780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:26.804 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:26.802+0000 7f4a37819780 -1 bdev(0x3800c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:26.804 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:26.802+0000 7f4a37819780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:44:27.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:44:27.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:27.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:44:27.283 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:44:27.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:27.387 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:44:27.388 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:44:27.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:27.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:27.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:27.389 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:27.402 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:27.401+0000 7f45a2a1d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:27.404 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:27.403+0000 7f45a2a1d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:27.405 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:27.404+0000 7f45a2a1d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:27.484 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:44:27.484 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:27.485 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:44:27.485 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:27.485 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:27.485 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:27.485 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:27.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:27.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:27.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:27.580 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:27.621 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:27.620+0000 7f45a2a1d780 -1 Falling back to public interface 2026-02-17T18:44:28.056 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:28.054+0000 7f45a2a1d780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:44:28.581 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:28.581 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:28.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:28.582 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:28.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:28.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:28.683 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3674389030,v1:127.0.0.1:6801/3674389030] [v2:127.0.0.1:6802/3674389030,v1:127.0.0.1:6803/3674389030] exists,up 328c154a-2ace-4276-806a-34df25ba08cb 2026-02-17T18:44:28.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:28.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:28.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:28.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1426: test_get_num_pgs: create_rbd_pool 2026-02-17T18:44:28.684 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:44:28.775 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:44:28.781 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:44:28.781 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:44:29.032 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:44:29.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:44:30.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1427: test_get_num_pgs: wait_for_clean 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:44:30.322 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:44:30.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:44:30.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:44:30.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:44:30.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:44:30.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:44:30.403 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:44:30.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:44:30.404 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:44:30.634 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:44:30.634 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:44:30.634 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:30.634 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:44:30.698 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:44:30.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:44:30.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:44:30.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:30.699 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:30.699 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:30.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:44:30.701 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:30.701 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:30.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:44:30.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:44:30.702 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:44:30.702 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:30.925 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:30.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:31.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:44:31.927 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:32.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:32.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:33.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:44:33.152 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:33.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:44:33.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:44:33.381 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:33.381 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:44:33.696 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:33.917 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:44:33.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:44:33.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:33.918 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:34.232 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:44:34.232 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:44:34.232 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:44:34.232 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: get_num_pgs 2026-02-17T18:44:34.232 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:34.232 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: local num_pgs=4 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1429: test_get_num_pgs: test 4 -gt 0 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1430: test_get_num_pgs: teardown td/ceph-helpers 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:34.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:34.651 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:34.651 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:34.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:34.652 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:34.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:34.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:34.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:34.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:34.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:34.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:34.655 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:34.655 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:34.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:34.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:34.660 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:34.660 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.660 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.660 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:34.662 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:34.662 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:34.662 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:44:34.662 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_op_scheduler td/ceph-helpers 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: test_get_op_scheduler: local dir=td/ceph-helpers 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: test_get_op_scheduler: setup td/ceph-helpers 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:34.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:34.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:34.665 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:34.665 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:34.665 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:34.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:34.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:34.666 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:34.666 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:34.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:34.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:34.667 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:34.667 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:34.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:34.668 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:34.670 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:34.670 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.670 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.670 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:34.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:34.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:34.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:44:34.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:44:34.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.672 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: test_get_op_scheduler: run_mon td/ceph-helpers a 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:44:34.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:44:34.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-17T18:44:34.692 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:44:34.692 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:34.692 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:34.693 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:34.693 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.693 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.693 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:34.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T18:44:34.716 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:44:34.717 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:44:34.717 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.718 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:34.719 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:34.719 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:44:34.760 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:44:34.760 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.761 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:34.762 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:34.762 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:44:34.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: test_get_op_scheduler: run_mgr td/ceph-helpers x 2026-02-17T18:44:34.802 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:44:34.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:44:34.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:44:34.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:44:34.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:44:34.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:34.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:34.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:34.901 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:34.901 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:34.901 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.902 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.902 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:34.902 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:34.902 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:34.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2322: test_get_op_scheduler: run_osd td/ceph-helpers 0 --osd_op_queue=wpq 2026-02-17T18:44:34.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:34.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:34.920 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=wpq 2026-02-17T18:44:34.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:34.922 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:34.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a9c2697f-8816-482c-9073-67b750a41afa 2026-02-17T18:44:34.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a9c2697f-8816-482c-9073-67b750a41afa' 2026-02-17T18:44:34.922 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 a9c2697f-8816-482c-9073-67b750a41afa 2026-02-17T18:44:34.922 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:34.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQASt5RpJ4t7NxAACBsct2GNgPgoRSXJ9g6NZw== 2026-02-17T18:44:34.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQASt5RpJ4t7NxAACBsct2GNgPgoRSXJ9g6NZw=="}' 2026-02-17T18:44:34.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a9c2697f-8816-482c-9073-67b750a41afa -i td/ceph-helpers/0/new.json 2026-02-17T18:44:35.037 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:35.044 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:35.045 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq --mkfs --key AQASt5RpJ4t7NxAACBsct2GNgPgoRSXJ9g6NZw== --osd-uuid a9c2697f-8816-482c-9073-67b750a41afa 2026-02-17T18:44:35.061 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.060+0000 7eff36e62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:35.062 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.062+0000 7eff36e62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:35.063 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.062+0000 7eff36e62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:35.064 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.063+0000 7eff36e62780 -1 bdev(0x2df6c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:35.064 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.063+0000 7eff36e62780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:44:35.469 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:44:35.469 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:35.470 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:44:35.470 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:44:35.470 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:35.573 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:44:35.574 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:44:35.574 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:35.575 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq 2026-02-17T18:44:35.575 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:35.575 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:35.589 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.588+0000 7fb0b3029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:35.590 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.589+0000 7fb0b3029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:35.591 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.590+0000 7fb0b3029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:44:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:44:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:35.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:35.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:35.672 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:35.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:35.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:35.770 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:35.832 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:35.830+0000 7fb0b3029780 -1 Falling back to public interface 2026-02-17T18:44:36.156 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:36.154+0000 7fb0b3029780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:44:36.771 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:36.771 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:36.771 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:36.771 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:36.772 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:36.772 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:36.868 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1935702769,v1:127.0.0.1:6801/1935702769] [v2:127.0.0.1:6802/1935702769,v1:127.0.0.1:6803/1935702769] exists,up a9c2697f-8816-482c-9073-67b750a41afa 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: get_op_scheduler 0 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=0 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 0 osd_op_queue 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-17T18:44:36.870 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.0.asok 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:36.871 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-osd.0.asok config get osd_op_queue 2026-02-17T18:44:36.919 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: test wpq = wpq 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2325: test_get_op_scheduler: run_osd td/ceph-helpers 1 --osd_op_queue=mclock_scheduler 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:36.920 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-17T18:44:36.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-17T18:44:36.922 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:36.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b9f5ec5e-80c5-47dd-bd39-72b4a69aad2d 2026-02-17T18:44:36.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b9f5ec5e-80c5-47dd-bd39-72b4a69aad2d' 2026-02-17T18:44:36.922 INFO:tasks.workunit.client.0.trial068.stdout:add osd1 b9f5ec5e-80c5-47dd-bd39-72b4a69aad2d 2026-02-17T18:44:36.922 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:36.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAUt5Rpep95NxAArv5Az3F7+d0zH8WwwiA01Q== 2026-02-17T18:44:36.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAUt5Rpep95NxAArv5Az3F7+d0zH8WwwiA01Q=="}' 2026-02-17T18:44:36.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b9f5ec5e-80c5-47dd-bd39-72b4a69aad2d -i td/ceph-helpers/1/new.json 2026-02-17T18:44:37.033 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:37.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-17T18:44:37.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQAUt5Rpep95NxAArv5Az3F7+d0zH8WwwiA01Q== --osd-uuid b9f5ec5e-80c5-47dd-bd39-72b4a69aad2d 2026-02-17T18:44:37.055 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.054+0000 7f9b00e27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:37.057 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.056+0000 7f9b00e27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:37.058 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.056+0000 7f9b00e27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:37.058 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.057+0000 7f9b00e27780 -1 bdev(0x2da4c00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:37.058 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.057+0000 7f9b00e27780 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-17T18:44:37.381 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-17T18:44:37.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:37.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T18:44:37.382 INFO:tasks.workunit.client.0.trial068.stdout:adding osd1 key to auth repository 2026-02-17T18:44:37.383 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:37.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T18:44:37.706 INFO:tasks.workunit.client.0.trial068.stdout:start osd.1 2026-02-17T18:44:37.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-17T18:44:37.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:37.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:37.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:37.721 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.719+0000 7f815be0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:37.722 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.721+0000 7f815be0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:37.723 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.722+0000 7f815be0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:37.853 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:37.851+0000 7f815be0b780 -1 Falling back to public interface 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:37.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:37.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:44:38.155 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:38.156 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:38.155+0000 7f815be0b780 -1 osd.1 0 log_to_monitors true 2026-02-17T18:44:39.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:39.156 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:39.157 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:39.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:39.158 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:39.158 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:44:39.382 INFO:tasks.workunit.client.0.trial068.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3583702566,v1:127.0.0.1:6811/3583702566] [v2:127.0.0.1:6812/3583702566,v1:127.0.0.1:6813/3583702566] exists,up b9f5ec5e-80c5-47dd-bd39-72b4a69aad2d 2026-02-17T18:44:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:39.384 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: get_op_scheduler 1 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=1 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 1 osd_op_queue 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=1 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-17T18:44:39.385 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.1 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-osd.1.asok 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:39.386 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-osd.1.asok config get osd_op_queue 2026-02-17T18:44:39.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: test mclock_scheduler = mclock_scheduler 2026-02-17T18:44:39.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: test_get_op_scheduler: teardown td/ceph-helpers 2026-02-17T18:44:39.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:39.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:39.431 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:39.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:39.432 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:39.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:39.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:39.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:39.538 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:39.539 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:39.539 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:39.540 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:39.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:39.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:39.541 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:39.542 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:39.542 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:39.543 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:39.543 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:39.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:39.544 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:39.545 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:39.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:39.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osd_id_used_by_pgs td/ceph-helpers 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1450: test_get_osd_id_used_by_pgs: local dir=td/ceph-helpers 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1452: test_get_osd_id_used_by_pgs: setup td/ceph-helpers 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:44:39.551 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:39.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:39.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:39.552 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:39.552 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:39.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:39.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:39.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:39.553 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:39.553 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:39.553 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:39.553 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:39.554 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:39.554 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:39.554 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:39.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:39.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:39.556 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:39.556 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:39.556 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:39.557 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:39.557 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:39.558 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:39.559 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.559 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:39.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:39.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:39.560 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:44:39.561 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:44:39.561 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.561 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:44:39.562 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:44:39.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:44:39.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:44:39.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:44:39.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1453: test_get_osd_id_used_by_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:39.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:44:39.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:44:39.563 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:44:39.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:44:39.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:44:39.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:39.581 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:44:39.581 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:39.581 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:39.581 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:39.582 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.582 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.582 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:39.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:39.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:44:39.607 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:44:39.607 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:39.607 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:39.607 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:44:39.607 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:44:39.608 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:39.609 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:44:39.649 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:44:39.650 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:39.650 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:39.650 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:44:39.650 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:44:39.651 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:39.651 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:39.651 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:39.651 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:39.651 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.652 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.652 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:39.652 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:39.652 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:44:39.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1454: test_get_osd_id_used_by_pgs: run_mgr td/ceph-helpers x 2026-02-17T18:44:39.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:44:39.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:44:39.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:44:39.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:44:39.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:44:39.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:39.790 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:39.791 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:39.791 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:39.791 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:39.791 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.791 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.791 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:39.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:39.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1455: test_get_osd_id_used_by_pgs: run_osd td/ceph-helpers 0 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:39.809 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:39.810 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:39.811 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:39.812 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:39.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=23a2dfb8-62fe-4189-af90-75324fe2002b 2026-02-17T18:44:39.812 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 23a2dfb8-62fe-4189-af90-75324fe2002b 2026-02-17T18:44:39.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 23a2dfb8-62fe-4189-af90-75324fe2002b' 2026-02-17T18:44:39.813 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:39.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAXt5RpW2DKMBAAmYL4VkeHCe6MsQwTPwxoXQ== 2026-02-17T18:44:39.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAXt5RpW2DKMBAAmYL4VkeHCe6MsQwTPwxoXQ=="}' 2026-02-17T18:44:39.820 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 23a2dfb8-62fe-4189-af90-75324fe2002b -i td/ceph-helpers/0/new.json 2026-02-17T18:44:39.922 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:39.928 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:39.929 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAXt5RpW2DKMBAAmYL4VkeHCe6MsQwTPwxoXQ== --osd-uuid 23a2dfb8-62fe-4189-af90-75324fe2002b 2026-02-17T18:44:39.945 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:39.944+0000 7fef03a4c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:39.947 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:39.946+0000 7fef03a4c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:39.948 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:39.947+0000 7fef03a4c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:39.948 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:39.947+0000 7fef03a4c780 -1 bdev(0x426ac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:39.948 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:39.947+0000 7fef03a4c780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:44:40.370 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:44:40.370 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:40.371 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:44:40.371 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:44:40.372 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:40.473 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:44:40.473 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:44:40.474 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:40.474 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:40.474 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:40.474 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:40.487 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:40.486+0000 7fc47264e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:40.488 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:40.487+0000 7fc47264e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:40.489 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:40.488+0000 7fc47264e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:40.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:44:40.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:40.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:44:40.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:40.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:40.572 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:40.573 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:40.573 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:40.573 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:40.573 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:40.671 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:40.733 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:40.732+0000 7fc47264e780 -1 Falling back to public interface 2026-02-17T18:44:41.066 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:41.064+0000 7fc47264e780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:44:41.672 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:41.673 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:41.673 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:41.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:41.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:41.674 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:41.773 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/282426167,v1:127.0.0.1:6801/282426167] [v2:127.0.0.1:6802/282426167,v1:127.0.0.1:6803/282426167] exists,up 23a2dfb8-62fe-4189-af90-75324fe2002b 2026-02-17T18:44:41.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:41.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:41.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:41.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1456: test_get_osd_id_used_by_pgs: create_rbd_pool 2026-02-17T18:44:41.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:44:41.867 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:44:41.873 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:44:41.874 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:44:42.151 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:44:42.159 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:44:43.161 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:44:43.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1457: test_get_osd_id_used_by_pgs: wait_for_clean 2026-02-17T18:44:43.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:44:43.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:44:43.441 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:44:43.441 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:44:43.441 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:44:43.442 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:44:43.442 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:44:43.442 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:44:43.442 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:44:43.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:44:43.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:44:43.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:44:43.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:44:43.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:44:43.523 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:44:43.746 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:44:43.747 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:44:43.747 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:43.747 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:44:43.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:44:43.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:44:43.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:44:43.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:43.812 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:43.812 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:43.813 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:44:43.813 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:43.814 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:43.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:44:43.814 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:44:43.815 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:44:43.815 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:44.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:44.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:45.050 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:44:45.050 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:45.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:45.277 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:46.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:44:46.279 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:46.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:44:46.506 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:44:46.507 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:46.507 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:46.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:44:46.819 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:44:46.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:44:46.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:44:46.819 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:44:46.820 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:44:46.820 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:44:46.820 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:47.038 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:44:47.038 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:44:47.038 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:47.039 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:47.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:44:47.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:44:47.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:44:47.357 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: get_osd_id_used_by_pgs 2026-02-17T18:44:47.358 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: uniq 2026-02-17T18:44:47.358 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: ceph --format json pg dump pgs 2026-02-17T18:44:47.358 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: jq '.pg_stats | .[] | .up[], .acting[]' 2026-02-17T18:44:47.358 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: sort 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: local osd_ids=0 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1459: test_get_osd_id_used_by_pgs: test 0 = 0 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1460: test_get_osd_id_used_by_pgs: teardown td/ceph-helpers 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:47.582 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:47.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:47.583 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:47.688 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:47.689 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:47.689 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:47.690 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:47.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:47.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:47.691 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:47.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:47.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:47.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:47.693 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:47.694 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:47.695 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:47.695 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:47.698 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:47.698 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.698 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.699 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:47.700 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:47.700 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osds td/ceph-helpers 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: test_get_osds: local dir=td/ceph-helpers 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1024: test_get_osds: setup td/ceph-helpers 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:47.701 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:47.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:47.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:47.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:47.702 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:47.702 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:47.703 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:47.704 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:47.704 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:47.704 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:47.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:47.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:47.705 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:47.705 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:47.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:47.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:47.707 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:47.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:47.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.708 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.708 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:47.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:47.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:47.709 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:44:47.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:44:47.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:44:47.711 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:44:47.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: test_get_osds: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:44:47.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-17T18:44:47.731 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:44:47.731 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:47.731 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:47.731 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:47.732 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.732 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.732 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:47.732 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T18:44:47.756 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:47.758 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:47.759 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:47.759 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.759 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.759 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:47.759 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:47.759 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:47.802 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:47.803 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:47.803 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.804 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.804 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:47.804 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:47.804 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:44:47.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1026: test_get_osds: run_mgr td/ceph-helpers x 2026-02-17T18:44:47.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:44:47.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:44:47.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:44:47.844 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:44:47.845 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:44:47.845 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:47.942 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:47.942 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:47.942 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:47.943 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:47.943 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.943 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.943 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:47.944 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:47.944 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:47.961 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: test_get_osds: run_osd td/ceph-helpers 0 2026-02-17T18:44:47.961 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:47.961 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:47.962 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:47.963 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=af4512fe-aa4f-438f-a094-f40014a2d7c7 2026-02-17T18:44:47.964 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 af4512fe-aa4f-438f-a094-f40014a2d7c7' 2026-02-17T18:44:47.965 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 af4512fe-aa4f-438f-a094-f40014a2d7c7 2026-02-17T18:44:47.965 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:47.971 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAft5RpUqrTORAA5vEykGvBK4QMFOvOFVTwVw== 2026-02-17T18:44:47.971 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAft5RpUqrTORAA5vEykGvBK4QMFOvOFVTwVw=="}' 2026-02-17T18:44:47.971 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new af4512fe-aa4f-438f-a094-f40014a2d7c7 -i td/ceph-helpers/0/new.json 2026-02-17T18:44:48.073 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:48.080 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:48.081 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAft5RpUqrTORAA5vEykGvBK4QMFOvOFVTwVw== --osd-uuid af4512fe-aa4f-438f-a094-f40014a2d7c7 2026-02-17T18:44:48.098 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.096+0000 7fa9a5e0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:48.099 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.098+0000 7fa9a5e0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:48.100 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.099+0000 7fa9a5e0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:48.100 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.099+0000 7fa9a5e0e780 -1 bdev(0x3d12c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:48.101 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.099+0000 7fa9a5e0e780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:44:48.484 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:44:48.485 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:48.485 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:44:48.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:44:48.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:48.588 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:44:48.589 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:44:48.589 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:48.590 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:48.590 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:48.590 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:48.604 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.602+0000 7efc80432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:48.605 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.604+0000 7efc80432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:48.606 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.605+0000 7efc80432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:48.689 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:44:48.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:48.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:44:48.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:48.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:48.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:48.690 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:48.690 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:48.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:48.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:48.789 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:48.806 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:48.805+0000 7efc80432780 -1 Falling back to public interface 2026-02-17T18:44:49.157 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:49.155+0000 7efc80432780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:44:49.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:49.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:49.791 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:49.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:49.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:49.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:44:49.888 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/428315418,v1:127.0.0.1:6801/428315418] [v2:127.0.0.1:6802/428315418,v1:127.0.0.1:6803/428315418] exists,up af4512fe-aa4f-438f-a094-f40014a2d7c7 2026-02-17T18:44:49.892 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:49.892 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:49.892 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:49.892 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1028: test_get_osds: run_osd td/ceph-helpers 1 2026-02-17T18:44:49.892 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:49.893 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:49.894 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7f8fac4-b9b7-4bf4-8fde-8fded2614d3d 2026-02-17T18:44:49.895 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c7f8fac4-b9b7-4bf4-8fde-8fded2614d3d' 2026-02-17T18:44:49.896 INFO:tasks.workunit.client.0.trial068.stdout:add osd1 c7f8fac4-b9b7-4bf4-8fde-8fded2614d3d 2026-02-17T18:44:49.896 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:49.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAht5RpEzCmNRAAUR0Z1MO1sTO4OspLzgA0fw== 2026-02-17T18:44:49.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAht5RpEzCmNRAAUR0Z1MO1sTO4OspLzgA0fw=="}' 2026-02-17T18:44:49.901 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7f8fac4-b9b7-4bf4-8fde-8fded2614d3d -i td/ceph-helpers/1/new.json 2026-02-17T18:44:50.007 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:50.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-17T18:44:50.013 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAht5RpEzCmNRAAUR0Z1MO1sTO4OspLzgA0fw== --osd-uuid c7f8fac4-b9b7-4bf4-8fde-8fded2614d3d 2026-02-17T18:44:50.028 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.027+0000 7f1c63843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:50.030 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.028+0000 7f1c63843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:50.030 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.029+0000 7f1c63843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:50.031 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.030+0000 7f1c63843780 -1 bdev(0x3e3cc00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:50.031 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.030+0000 7f1c63843780 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-17T18:44:50.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-17T18:44:50.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:44:50.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-17T18:44:50.356 INFO:tasks.workunit.client.0.trial068.stdout:adding osd1 key to auth repository 2026-02-17T18:44:50.356 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:44:50.674 INFO:tasks.workunit.client.0.trial068.stdout:start osd.1 2026-02-17T18:44:50.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-17T18:44:50.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:44:50.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:44:50.675 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:44:50.676 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:44:50.688 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.686+0000 7f7786ffb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:50.689 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.688+0000 7f7786ffb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:50.690 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.689+0000 7f7786ffb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:50.820 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:50.818+0000 7f7786ffb780 -1 Falling back to public interface 2026-02-17T18:44:50.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-17T18:44:50.903 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:44:50.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-17T18:44:50.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:44:50.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:44:50.904 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:50.904 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:50.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:44:50.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:50.905 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:44:51.092 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:51.091+0000 7f7786ffb780 -1 osd.1 0 log_to_monitors true 2026-02-17T18:44:51.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:44:52.135 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:44:52.135 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:44:52.136 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:44:52.136 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:44:52.136 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:44:52.136 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-17T18:44:52.365 INFO:tasks.workunit.client.0.trial068.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3169821902,v1:127.0.0.1:6811/3169821902] [v2:127.0.0.1:6812/3169821902,v1:127.0.0.1:6813/3169821902] exists,up c7f8fac4-b9b7-4bf4-8fde-8fded2614d3d 2026-02-17T18:44:52.366 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:44:52.366 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:44:52.366 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:44:52.366 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1029: test_get_osds: create_rbd_pool 2026-02-17T18:44:52.366 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:44:52.586 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:44:52.593 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:44:52.593 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:44:52.841 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:44:52.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:44:53.850 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:44:54.128 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1030: test_get_osds: wait_for_clean 2026-02-17T18:44:54.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:44:54.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:44:54.129 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:44:54.129 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:44:54.129 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:44:54.129 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:44:54.129 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:44:54.130 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:44:54.130 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:44:54.210 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:44:54.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:44:54.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:44:54.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:44:54.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:44:54.211 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:44:54.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-17T18:44:54.434 INFO:tasks.workunit.client.0.trial068.stderr:1' 2026-02-17T18:44:54.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:44:54.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:54.434 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:44:54.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:44:54.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:44:54.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:44:54.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:44:54.497 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-17T18:44:54.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-17T18:44:54.562 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-17T18:44:54.562 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483 1-38654705667' 2026-02-17T18:44:54.562 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:54.562 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:54.562 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:54.562 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:44:54.563 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:44:54.563 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:54.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:44:54.564 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:44:54.564 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:44:54.565 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:54.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:44:54.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:44:55.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:44:55.791 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:44:56.016 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-17T18:44:56.016 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:44:56.016 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705667 2026-02-17T18:44:56.017 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:44:56.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-17T18:44:56.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-17T18:44:56.018 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:44:56.019 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705667 2026-02-17T18:44:56.019 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-17T18:44:56.019 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.1 seq 38654705667 2026-02-17T18:44:56.019 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T18:44:56.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-17T18:44:56.244 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:44:56.245 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:56.245 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:56.566 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:44:56.566 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:44:56.566 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:44:56.566 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:44:56.567 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:44:56.567 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:44:56.567 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:44:56.567 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:44:56.788 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:44:56.788 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:44:56.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:44:56.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:44:57.101 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:44:57.101 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:44:57.101 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:44:57.101 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: test_get_osds: create_rbd_pool 2026-02-17T18:44:57.102 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:44:57.371 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:44:57.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:44:57.378 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:44:57.648 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:44:57.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:44:58.658 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:44:58.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: get_osds rbd GROUP 2026-02-17T18:44:58.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=rbd 2026-02-17T18:44:58.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: grep --quiet '^[0-1] [0-1]$' 2026-02-17T18:44:58.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=GROUP 2026-02-17T18:44:58.940 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map rbd GROUP 2026-02-17T18:44:58.940 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-17T18:44:59.170 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr:0' 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1033: test_get_osds: teardown td/ceph-helpers 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:59.171 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:59.172 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:59.172 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:59.172 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:59.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:59.278 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:59.279 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:59.279 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:59.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:59.281 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:59.281 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:59.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:59.282 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:59.283 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:59.283 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:59.284 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:59.285 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:59.285 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:59.290 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:59.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.291 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:59.293 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:59.293 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:59.293 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:44:59.293 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_pg td/ceph-helpers 2026-02-17T18:44:59.293 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1091: test_get_pg: local dir=td/ceph-helpers 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: test_get_pg: setup td/ceph-helpers 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:44:59.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:44:59.295 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:44:59.295 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:44:59.296 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:44:59.296 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:44:59.296 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:44:59.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:44:59.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:44:59.297 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:44:59.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:59.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:44:59.297 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:44:59.297 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:44:59.298 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:44:59.299 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:44:59.299 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:44:59.300 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:44:59.300 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.300 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.300 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:44:59.301 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:44:59.301 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:44:59.301 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:44:59.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:44:59.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:44:59.305 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:44:59.305 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:44:59.305 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:44:59.305 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:44:59.305 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: test_get_pg: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:59.305 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:44:59.306 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:44:59.306 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:44:59.306 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:44:59.306 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:44:59.306 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:59.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:44:59.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:59.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:59.324 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:59.324 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.324 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:59.324 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:44:59.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:44:59.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:44:59.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:59.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:59.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:44:59.348 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:44:59.349 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:59.349 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:59.349 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:59.349 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:59.349 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.349 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.349 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:59.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:59.350 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:44:59.393 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:44:59.394 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.394 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.394 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:44:59.394 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:44:59.394 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:44:59.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1095: test_get_pg: run_mgr td/ceph-helpers x 2026-02-17T18:44:59.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:44:59.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:44:59.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:44:59.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:44:59.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:44:59.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:44:59.538 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:44:59.539 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:59.539 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:59.539 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:59.539 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.539 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.539 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:59.540 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:59.540 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:44:59.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: test_get_pg: run_osd td/ceph-helpers 0 2026-02-17T18:44:59.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:44:59.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:44:59.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:44:59.560 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:44:59.561 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:44:59.562 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:44:59.562 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fd97b050-a63d-4d85-b054-9604c4a5766e 2026-02-17T18:44:59.562 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fd97b050-a63d-4d85-b054-9604c4a5766e' 2026-02-17T18:44:59.562 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 fd97b050-a63d-4d85-b054-9604c4a5766e 2026-02-17T18:44:59.562 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:44:59.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQArt5RpNCvNIRAADuTHts5kq8a+hoJHzaYkzw== 2026-02-17T18:44:59.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQArt5RpNCvNIRAADuTHts5kq8a+hoJHzaYkzw=="}' 2026-02-17T18:44:59.568 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fd97b050-a63d-4d85-b054-9604c4a5766e -i td/ceph-helpers/0/new.json 2026-02-17T18:44:59.669 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:44:59.676 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:44:59.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQArt5RpNCvNIRAADuTHts5kq8a+hoJHzaYkzw== --osd-uuid fd97b050-a63d-4d85-b054-9604c4a5766e 2026-02-17T18:44:59.693 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:59.692+0000 7f991d62d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:59.694 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:59.693+0000 7f991d62d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:59.695 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:59.694+0000 7f991d62d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:44:59.696 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:59.694+0000 7f991d62d780 -1 bdev(0x42d6c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:44:59.696 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:44:59.694+0000 7f991d62d780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:45:00.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:45:00.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:45:00.143 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:45:00.143 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:45:00.143 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:45:00.247 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:45:00.247 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:45:00.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:00.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:45:00.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:45:00.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:45:00.262 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:00.261+0000 7f109662d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:00.264 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:00.263+0000 7f109662d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:00.265 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:00.264+0000 7f109662d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:00.345 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:45:00.345 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:00.345 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:00.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:00.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:00.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:00.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:00.346 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:00.347 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:00.347 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:00.443 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:00.505 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:00.504+0000 7f109662d780 -1 Falling back to public interface 2026-02-17T18:45:00.895 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:00.893+0000 7f109662d780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:45:01.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:01.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:01.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:01.445 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:45:01.446 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:01.446 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:01.547 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2504093187,v1:127.0.0.1:6801/2504093187] [v2:127.0.0.1:6802/2504093187,v1:127.0.0.1:6803/2504093187] exists,up fd97b050-a63d-4d85-b054-9604c4a5766e 2026-02-17T18:45:01.548 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:01.548 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:01.548 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:01.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1097: test_get_pg: create_rbd_pool 2026-02-17T18:45:01.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:45:01.642 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:45:01.649 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:45:01.649 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:45:01.923 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:45:01.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:45:02.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:45:03.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1098: test_get_pg: wait_for_clean 2026-02-17T18:45:03.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:45:03.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:45:03.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:45:03.212 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:45:03.212 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:03.213 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:03.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:03.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:03.213 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:03.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:45:03.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:45:03.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:45:03.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:45:03.294 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:45:03.294 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:45:03.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:45:03.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:45:03.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:45:03.522 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:45:03.585 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:45:03.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:45:03.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:45:03.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:45:03.586 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:03.586 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:45:03.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:45:03.587 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:03.587 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:45:03.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:45:03.588 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:45:03.589 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:45:03.589 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:03.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:45:03.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:04.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:45:04.825 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:05.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:45:05.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:06.059 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:45:06.060 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:06.290 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:45:06.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:45:06.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:06.291 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:06.609 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:45:06.609 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:45:06.609 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:45:06.610 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:45:06.610 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:45:06.610 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:45:06.610 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:45:06.610 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:45:06.830 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:45:06.831 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:45:06.831 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:06.831 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:07.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:45:07.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:45:07.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:45:07.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: get_pg rbd GROUP 2026-02-17T18:45:07.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-17T18:45:07.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: grep --quiet '^[0-9]\.[0-9a-f][0-9a-f]*$' 2026-02-17T18:45:07.143 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=GROUP 2026-02-17T18:45:07.143 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd GROUP 2026-02-17T18:45:07.143 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-17T18:45:07.373 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1100: test_get_pg: teardown td/ceph-helpers 2026-02-17T18:45:07.373 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:07.373 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:07.373 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:07.373 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:07.374 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:07.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:07.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:07.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:07.481 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:07.481 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:07.482 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:07.482 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:07.483 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:07.483 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:07.483 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:07.484 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:07.484 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:07.484 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:07.484 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:07.485 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:07.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:07.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:07.489 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:07.489 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.490 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.490 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_primary td/ceph-helpers 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: test_get_primary: local dir=td/ceph-helpers 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: test_get_primary: setup td/ceph-helpers 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:07.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:07.493 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:07.493 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:07.494 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:07.494 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:07.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:07.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:07.496 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:07.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:07.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:07.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:07.498 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:07.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:07.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:07.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:07.500 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:07.500 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.500 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.500 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:07.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:07.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:07.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:45:07.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:45:07.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.502 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:45:07.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:45:07.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:45:07.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:45:07.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:45:07.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: test_get_primary: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:07.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:45:07.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:45:07.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:45:07.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:45:07.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:45:07.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:07.522 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:45:07.522 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:07.522 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:07.523 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:07.523 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.523 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.523 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:07.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:07.547 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:45:07.548 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.549 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:07.550 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:07.550 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:45:07.592 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:45:07.592 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:07.592 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:07.592 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:45:07.593 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:07.594 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:45:07.636 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: test_get_primary: local osd=0 2026-02-17T18:45:07.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: test_get_primary: run_mgr td/ceph-helpers x 2026-02-17T18:45:07.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:45:07.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:45:07.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:45:07.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:45:07.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:45:07.637 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:45:07.739 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:45:07.739 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:07.740 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:07.740 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:07.740 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.740 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.740 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:07.740 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:07.741 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:07.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1201: test_get_primary: run_osd td/ceph-helpers 0 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:45:07.760 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:07.761 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:45:07.762 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:45:07.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d8ca42b6-b923-46b0-aa28-95573158ec5f 2026-02-17T18:45:07.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d8ca42b6-b923-46b0-aa28-95573158ec5f' 2026-02-17T18:45:07.763 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 d8ca42b6-b923-46b0-aa28-95573158ec5f 2026-02-17T18:45:07.763 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:45:07.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAzt5RpWpDELRAAjeLmzicGNcMW4zid/g5O1A== 2026-02-17T18:45:07.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAzt5RpWpDELRAAjeLmzicGNcMW4zid/g5O1A=="}' 2026-02-17T18:45:07.770 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d8ca42b6-b923-46b0-aa28-95573158ec5f -i td/ceph-helpers/0/new.json 2026-02-17T18:45:07.874 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:07.880 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:45:07.881 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAzt5RpWpDELRAAjeLmzicGNcMW4zid/g5O1A== --osd-uuid d8ca42b6-b923-46b0-aa28-95573158ec5f 2026-02-17T18:45:07.897 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:07.895+0000 7f25d6652780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:07.898 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:07.897+0000 7f25d6652780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:07.899 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:07.898+0000 7f25d6652780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:07.899 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:07.898+0000 7f25d6652780 -1 bdev(0x43bec00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:45:07.900 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:07.898+0000 7f25d6652780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:45:08.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:45:08.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:45:08.303 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:45:08.303 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:45:08.303 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:45:08.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:45:08.405 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:45:08.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:08.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:45:08.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:45:08.406 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:45:08.419 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:08.418+0000 7f558b667780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:08.420 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:08.419+0000 7f558b667780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:08.421 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:08.420+0000 7f558b667780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:08.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:45:08.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:08.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:08.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:08.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:08.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:08.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:08.507 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:08.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:08.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:08.605 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:08.638 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:08.637+0000 7f558b667780 -1 Falling back to public interface 2026-02-17T18:45:08.970 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:08.969+0000 7f558b667780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:45:09.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:09.606 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:09.607 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:45:09.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:09.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:09.607 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:09.705 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1276447314,v1:127.0.0.1:6801/1276447314] [v2:127.0.0.1:6802/1276447314,v1:127.0.0.1:6803/1276447314] exists,up d8ca42b6-b923-46b0-aa28-95573158ec5f 2026-02-17T18:45:09.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:09.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:09.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:09.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: test_get_primary: create_rbd_pool 2026-02-17T18:45:09.706 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:45:09.797 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:45:09.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:45:09.803 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:45:10.034 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:45:10.042 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:45:11.044 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:45:11.323 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: test_get_primary: wait_for_clean 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:11.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:11.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:45:11.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:45:11.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:45:11.404 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:45:11.405 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:45:11.405 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:45:11.633 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:45:11.633 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:45:11.634 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:45:11.634 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:45:11.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:45:11.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:45:11.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:45:11.697 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:45:11.697 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:11.697 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:45:11.698 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:45:11.699 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:11.699 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:45:11.700 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:45:11.700 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:45:11.700 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:45:11.700 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:11.926 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:45:11.926 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:45:11.927 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:11.927 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:12.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:45:12.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:45:12.239 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:45:12.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:45:12.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:45:12.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:45:12.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:45:12.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:45:12.462 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:45:12.463 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:45:12.463 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:12.463 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:12.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:45:12.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:45:12.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:45:12.777 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: get_primary rbd GROUP 2026-02-17T18:45:12.777 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-17T18:45:12.777 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-17T18:45:12.777 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-17T18:45:12.777 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-17T18:45:13.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: test 0 = 0 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1205: test_get_primary: teardown td/ceph-helpers 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:13.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:13.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:13.109 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:13.110 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:13.110 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:13.111 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:13.111 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:13.112 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:13.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:13.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:13.113 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:13.114 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:13.114 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:13.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:13.115 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:13.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:13.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:13.119 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:13.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:13.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:13.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:13.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:45:13.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_timeout_delays td/ceph-helpers 2026-02-17T18:45:13.120 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: get_timeout_delays 1 2026-02-17T18:45:13.121 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.121 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.121 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.121 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.121 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.130 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: test '1 ' = '1 ' 2026-02-17T18:45:13.131 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: get_timeout_delays 5 2026-02-17T18:45:13.132 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.132 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.132 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.149 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: test '1 2 2 ' = '1 2 2 ' 2026-02-17T18:45:13.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: get_timeout_delays 6 2026-02-17T18:45:13.150 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.150 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.150 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.167 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: test '1 2 3 ' = '1 2 3 ' 2026-02-17T18:45:13.167 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: get_timeout_delays 7 2026-02-17T18:45:13.168 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.168 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.168 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.190 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: test '1 2 4 ' = '1 2 4 ' 2026-02-17T18:45:13.190 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: get_timeout_delays 8 2026-02-17T18:45:13.191 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.191 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.191 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.191 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.191 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.215 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: test '1 2 4 1 ' = '1 2 4 1 ' 2026-02-17T18:45:13.215 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: get_timeout_delays 1 .1 2026-02-17T18:45:13.216 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.216 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.216 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.216 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.216 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.239 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: test '0.1 0.2 0.4 0.3 ' = '0.1 0.2 0.4 0.3 ' 2026-02-17T18:45:13.240 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: get_timeout_delays 1.5 .1 2026-02-17T18:45:13.240 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.241 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.241 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.241 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.241 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.269 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 ' = '0.1 0.2 0.4 0.8 ' 2026-02-17T18:45:13.270 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: get_timeout_delays 5 .1 2026-02-17T18:45:13.271 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.271 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.271 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.308 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 1.9 ' = '0.1 0.2 0.4 0.8 1.6 1.9 ' 2026-02-17T18:45:13.308 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: get_timeout_delays 6 .1 2026-02-17T18:45:13.309 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.309 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.309 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.309 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.309 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.345 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 2.9 ' = '0.1 0.2 0.4 0.8 1.6 2.9 ' 2026-02-17T18:45:13.346 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: get_timeout_delays 6.3 .1 2026-02-17T18:45:13.347 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.347 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.347 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.347 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.347 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.382 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 ' = '0.1 0.2 0.4 0.8 1.6 3.2 ' 2026-02-17T18:45:13.383 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: get_timeout_delays 20 .1 2026-02-17T18:45:13.383 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.383 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.384 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.384 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.384 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' 2026-02-17T18:45:13.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: get_timeout_delays 300 .1 0 2026-02-17T18:45:13.435 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.435 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.510 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' 2026-02-17T18:45:13.511 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: get_timeout_delays 300 .1 10 2026-02-17T18:45:13.511 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:13.512 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:13.512 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:13.512 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:13.512 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_is_clean td/ceph-helpers 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1552: test_is_clean: local dir=td/ceph-helpers 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1554: test_is_clean: setup td/ceph-helpers 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:13.737 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:13.739 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:13.739 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:13.740 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:13.740 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:13.741 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:13.741 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:13.741 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:13.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:13.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:13.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:13.742 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:13.743 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:13.744 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:13.744 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:13.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:13.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:13.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:13.745 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:13.746 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:13.746 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:13.746 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:45:13.747 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:45:13.747 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:13.747 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:13.747 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:45:13.748 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1555: test_is_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:45:13.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:13.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:45:13.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:13.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:13.767 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:13.767 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:13.767 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:13.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:13.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:13.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:13.792 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:13.793 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:13.793 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:13.793 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:13.793 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:13.793 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:13.793 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:13.838 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:13.839 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:13.839 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:13.839 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:13.840 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:13.840 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:13.840 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:45:13.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1556: test_is_clean: run_mgr td/ceph-helpers x 2026-02-17T18:45:13.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:45:13.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:45:13.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:45:13.882 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:45:13.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:45:13.883 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:45:13.980 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:45:13.980 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:13.981 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:13.981 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:13.981 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:13.982 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:13.982 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:13.982 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:13.982 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:14.000 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1557: test_is_clean: run_osd td/ceph-helpers 0 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:45:14.001 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:45:14.002 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:45:14.003 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:45:14.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fdb794eb-19a5-4627-b09e-54746334094d 2026-02-17T18:45:14.004 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 fdb794eb-19a5-4627-b09e-54746334094d 2026-02-17T18:45:14.005 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fdb794eb-19a5-4627-b09e-54746334094d' 2026-02-17T18:45:14.005 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:45:14.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA6t5RpsKefABAAacJjVBiXuWSpNhJLdOFftA== 2026-02-17T18:45:14.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA6t5RpsKefABAAacJjVBiXuWSpNhJLdOFftA=="}' 2026-02-17T18:45:14.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fdb794eb-19a5-4627-b09e-54746334094d -i td/ceph-helpers/0/new.json 2026-02-17T18:45:14.113 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:14.119 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:45:14.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA6t5RpsKefABAAacJjVBiXuWSpNhJLdOFftA== --osd-uuid fdb794eb-19a5-4627-b09e-54746334094d 2026-02-17T18:45:14.136 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.135+0000 7fd074419780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:14.137 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.136+0000 7fd074419780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:14.138 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.137+0000 7fd074419780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:14.139 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.137+0000 7fd074419780 -1 bdev(0x42dcc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:45:14.139 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.137+0000 7fd074419780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:45:14.547 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:45:14.547 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:45:14.548 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:45:14.548 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:45:14.548 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:45:14.652 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:45:14.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:45:14.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:14.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:45:14.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:45:14.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:45:14.667 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.666+0000 7fb8f7a37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:14.669 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.668+0000 7fb8f7a37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:14.670 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.669+0000 7fb8f7a37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:14.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:45:14.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:14.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:14.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:14.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:14.752 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:14.752 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:14.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:14.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:14.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:14.846 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:14.932 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:14.931+0000 7fb8f7a37780 -1 Falling back to public interface 2026-02-17T18:45:15.262 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:15.261+0000 7fb8f7a37780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:45:15.848 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:15.848 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:15.848 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:45:15.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:15.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:15.849 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:15.949 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2142939401,v1:127.0.0.1:6801/2142939401] [v2:127.0.0.1:6802/2142939401,v1:127.0.0.1:6803/2142939401] exists,up fdb794eb-19a5-4627-b09e-54746334094d 2026-02-17T18:45:15.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:15.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:15.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:15.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1558: test_is_clean: create_rbd_pool 2026-02-17T18:45:15.950 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:45:16.043 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:45:16.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:45:16.049 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:45:16.337 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:45:16.345 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:45:17.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:45:17.626 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1559: test_is_clean: wait_for_clean 2026-02-17T18:45:17.626 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:45:17.626 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:45:17.626 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:45:17.627 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:45:17.627 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:17.628 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:17.628 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:17.628 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:17.628 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:17.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:45:17.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:45:17.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:45:17.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:45:17.711 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:45:17.711 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:45:17.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:45:17.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:45:17.939 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:45:17.940 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:45:18.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:45:18.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:45:18.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:45:18.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:45:18.004 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:18.004 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:45:18.004 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:45:18.005 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:18.005 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:45:18.006 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:45:18.006 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:45:18.007 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:45:18.007 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:18.231 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:45:18.231 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:19.232 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:45:19.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:19.458 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:45:19.458 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:20.459 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:45:20.460 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:20.685 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:45:20.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:45:20.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:20.686 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:21.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:45:21.003 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:45:21.003 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:45:21.003 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:45:21.003 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:45:21.004 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:45:21.004 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:45:21.004 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:45:21.235 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:45:21.235 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:45:21.235 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:21.236 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:21.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:45:21.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:45:21.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:45:21.549 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1560: test_is_clean: is_clean 2026-02-17T18:45:21.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: get_num_pgs 2026-02-17T18:45:21.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:21.549 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:21.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: num_pgs=4 2026-02-17T18:45:21.864 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1547: is_clean: test 4 '!=' 0 2026-02-17T18:45:21.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: get_num_active_clean 2026-02-17T18:45:21.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:45:21.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:45:21.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:45:21.864 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:45:21.865 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: test 4 = 4 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1561: test_is_clean: teardown td/ceph-helpers 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:22.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:22.084 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:22.084 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:22.189 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:22.190 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:22.190 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:22.190 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:22.191 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:22.191 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:22.192 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:22.192 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:22.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:22.193 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:22.193 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:22.194 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:22.195 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:22.195 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:22.198 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:22.198 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.198 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.199 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:22.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:22.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemon td/ceph-helpers 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: test_kill_daemon: local dir=td/ceph-helpers 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:267: test_kill_daemon: setup td/ceph-helpers 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:22.201 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:22.202 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:22.202 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:22.202 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:22.202 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:22.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:22.203 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:22.204 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:22.204 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:22.204 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:22.205 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:22.205 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:22.205 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:22.206 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:22.206 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:22.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:22.207 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:22.208 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:22.208 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.208 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.208 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:22.209 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:22.209 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:22.209 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:45:22.210 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:45:22.210 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.210 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.211 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:45:22.211 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:45:22.212 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: test_kill_daemon: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:45:22.213 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:22.231 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:45:22.232 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:22.232 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:22.232 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:22.233 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.233 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:22.233 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:22.257 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:45:22.258 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:45:22.258 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:22.258 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:22.258 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:45:22.258 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:45:22.259 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:22.259 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:22.259 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:22.260 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:22.260 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.260 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.260 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:22.260 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:22.260 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:45:22.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:45:22.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:22.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:22.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:22.304 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:45:22.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: test_kill_daemon: run_mgr td/ceph-helpers x 2026-02-17T18:45:22.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:45:22.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:45:22.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:45:22.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:45:22.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:45:22.347 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:45:22.447 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:45:22.447 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:22.447 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:22.448 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:22.448 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.448 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.448 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:22.448 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:22.449 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:270: test_kill_daemon: run_osd td/ceph-helpers 0 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:45:22.466 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:22.467 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:45:22.468 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:45:22.469 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:45:22.469 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:45:22.469 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:45:22.469 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:45:22.469 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:45:22.470 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fff26824-0a42-4eb7-b61f-4ab7c97d7fd4 2026-02-17T18:45:22.470 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fff26824-0a42-4eb7-b61f-4ab7c97d7fd4' 2026-02-17T18:45:22.470 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 fff26824-0a42-4eb7-b61f-4ab7c97d7fd4 2026-02-17T18:45:22.470 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:45:22.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBCt5RprKpYHBAAAfjyCspqbr8fY2Djln2+Ew== 2026-02-17T18:45:22.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBCt5RprKpYHBAAAfjyCspqbr8fY2Djln2+Ew=="}' 2026-02-17T18:45:22.477 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fff26824-0a42-4eb7-b61f-4ab7c97d7fd4 -i td/ceph-helpers/0/new.json 2026-02-17T18:45:22.580 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:22.586 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:45:22.587 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBCt5RprKpYHBAAAfjyCspqbr8fY2Djln2+Ew== --osd-uuid fff26824-0a42-4eb7-b61f-4ab7c97d7fd4 2026-02-17T18:45:22.604 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:22.602+0000 7fd129246780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:22.605 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:22.604+0000 7fd129246780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:22.606 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:22.605+0000 7fd129246780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:22.606 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:22.605+0000 7fd129246780 -1 bdev(0x250ac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:45:22.607 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:22.605+0000 7fd129246780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:45:23.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:45:23.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:45:23.044 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:45:23.045 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:45:23.045 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:45:23.148 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:45:23.149 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:45:23.149 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:23.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:45:23.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:45:23.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:45:23.164 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:23.162+0000 7fe02dc5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:23.165 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:23.164+0000 7fe02dc5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:23.166 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:23.165+0000 7fe02dc5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:23.247 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:45:23.247 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:23.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:23.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:23.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:23.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:23.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:23.248 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:23.345 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:23.371 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:23.370+0000 7fe02dc5b780 -1 Falling back to public interface 2026-02-17T18:45:23.671 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:23.669+0000 7fe02dc5b780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:45:24.347 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:24.347 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:24.347 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:45:24.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:24.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:24.348 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:24.446 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3974121670,v1:127.0.0.1:6801/3974121670] [v2:127.0.0.1:6802/3974121670,v1:127.0.0.1:6803/3974121670] exists,up fff26824-0a42-4eb7-b61f-4ab7c97d7fd4 2026-02-17T18:45:24.447 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:24.447 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:24.447 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:24.447 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:272: test_kill_daemon: name_prefix=osd 2026-02-17T18:45:24.447 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: find td/ceph-helpers 2026-02-17T18:45:24.448 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: grep 'osd[^/]*\.pid' 2026-02-17T18:45:24.448 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-17T18:45:24.448 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:280: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid 0 1 2026-02-17T18:45:24.449 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-17T18:45:24.450 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=99837 2026-02-17T18:45:24.450 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=0 2026-02-17T18:45:24.450 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local delays=1 2026-02-17T18:45:24.450 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T18:45:24.450 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:24.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 99837 2026-02-17T18:45:24.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T18:45:24.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T18:45:24.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 1 2026-02-17T18:45:25.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:25.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 99837 2026-02-17T18:45:25.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T18:45:25.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T18:45:25.452 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0 2026-02-17T18:45:25.452 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 1 2026-02-17T18:45:25.452 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:284: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid TERM 2026-02-17T18:45:25.452 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=99837 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 99837 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T18:45:25.454 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-17T18:45:25.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:25.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 99837 2026-02-17T18:45:25.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-17T18:45:25.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-17T18:45:25.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-17T18:45:25.555 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:287: test_kill_daemon: name_prefix=mgr 2026-02-17T18:45:25.556 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: find td/ceph-helpers 2026-02-17T18:45:25.556 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: grep 'mgr[^/]*\.pid' 2026-02-17T18:45:25.556 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-17T18:45:25.556 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:292: test_kill_daemon: kill_daemon td/ceph-helpers/mgr.x.pid TERM 2026-02-17T18:45:25.557 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mgr.x.pid 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=98982 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 98982 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T18:45:25.558 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T18:45:25.559 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-17T18:45:25.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:25.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 98982 2026-02-17T18:45:25.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-17T18:45:25.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-17T18:45:25.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-17T18:45:25.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:295: test_kill_daemon: name_prefix=mon 2026-02-17T18:45:25.660 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: find td/ceph-helpers 2026-02-17T18:45:25.660 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: grep 'mon[^/]*\.pid' 2026-02-17T18:45:25.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-17T18:45:25.661 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:300: test_kill_daemon: kill_daemon td/ceph-helpers/mon.a.pid TERM 2026-02-17T18:45:25.662 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mon.a.pid 2026-02-17T18:45:25.663 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=98897 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 98897 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-17T18:45:25.664 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-17T18:45:25.764 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-17T18:45:25.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 98897 2026-02-17T18:45:25.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-17T18:45:25.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-17T18:45:25.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-17T18:45:25.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:301: test_kill_daemon: timeout 5 ceph status 2026-02-17T18:45:30.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:304: test_kill_daemon: teardown td/ceph-helpers 2026-02-17T18:45:30.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:30.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:30.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:30.769 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:30.769 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:30.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:30.770 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:30.770 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:30.773 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:30.774 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:30.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:30.775 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:30.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:30.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:30.776 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:30.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:30.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:30.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:30.778 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:30.778 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:30.779 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:30.779 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:30.782 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:30.782 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:30.782 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:30.782 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:30.784 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:30.784 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:30.784 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:45:30.784 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemons td/ceph-helpers 2026-02-17T18:45:30.784 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:357: test_kill_daemons: local dir=td/ceph-helpers 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:358: test_kill_daemons: setup td/ceph-helpers 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:30.785 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:30.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:30.786 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:30.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:30.787 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:30.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:30.787 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:30.788 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:30.788 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:30.788 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:30.789 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:30.789 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:30.790 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:30.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:30.791 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:30.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:30.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:30.792 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:30.792 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:30.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:30.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:30.793 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:45:30.794 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:45:30.794 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:30.794 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:30.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:45:30.795 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:45:30.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:45:30.796 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:359: test_kill_daemons: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:45:30.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:30.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:45:30.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:30.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:30.817 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:30.817 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:30.818 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:30.818 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:30.818 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:30.841 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:45:30.842 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:45:30.842 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:30.842 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:30.843 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:45:30.843 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:45:30.844 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:30.844 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:30.844 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:30.844 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:30.844 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:30.844 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:30.844 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:30.845 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:30.845 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:45:30.888 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:45:30.888 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:30.888 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:30.888 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:45:30.888 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:45:30.888 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:30.888 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:30.889 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:30.889 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:30.890 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:30.890 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:30.890 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:30.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:30.890 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:45:30.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:360: test_kill_daemons: run_mgr td/ceph-helpers x 2026-02-17T18:45:30.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:45:30.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:45:30.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:45:30.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:45:30.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:45:30.932 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:45:31.035 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:45:31.035 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:31.036 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:31.036 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:31.036 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:31.036 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:31.036 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:31.037 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:31.037 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:361: test_kill_daemons: run_osd td/ceph-helpers 0 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:45:31.054 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:31.055 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:45:31.056 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:45:31.057 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:45:31.057 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:45:31.057 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:45:31.057 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:45:31.057 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=acdbe1ca-ddcb-4aa0-a6a7-eafea3bad582 2026-02-17T18:45:31.058 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 acdbe1ca-ddcb-4aa0-a6a7-eafea3bad582' 2026-02-17T18:45:31.058 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 acdbe1ca-ddcb-4aa0-a6a7-eafea3bad582 2026-02-17T18:45:31.058 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:45:31.065 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBLt5RpOG/ZAxAAAFda1Ulu3tRVBGhBbOmVHA== 2026-02-17T18:45:31.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBLt5RpOG/ZAxAAAFda1Ulu3tRVBGhBbOmVHA=="}' 2026-02-17T18:45:31.066 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new acdbe1ca-ddcb-4aa0-a6a7-eafea3bad582 -i td/ceph-helpers/0/new.json 2026-02-17T18:45:31.171 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:31.178 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:45:31.179 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLt5RpOG/ZAxAAAFda1Ulu3tRVBGhBbOmVHA== --osd-uuid acdbe1ca-ddcb-4aa0-a6a7-eafea3bad582 2026-02-17T18:45:31.194 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.193+0000 7fde62a4e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:31.195 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.194+0000 7fde62a4e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:31.196 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.195+0000 7fde62a4e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:31.197 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.196+0000 7fde62a4e780 -1 bdev(0x2a8ac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:45:31.197 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.196+0000 7fde62a4e780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:45:31.622 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:45:31.623 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:45:31.624 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:45:31.624 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:45:31.624 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:45:31.726 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:45:31.727 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:45:31.727 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:31.727 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:45:31.727 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:45:31.727 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:45:31.741 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.740+0000 7f6d0b038780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:31.742 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.741+0000 7f6d0b038780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:31.743 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.742+0000 7f6d0b038780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:31.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:45:31.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:31.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:31.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:31.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:31.823 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:31.824 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:31.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:31.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:31.824 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:31.920 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:31.962 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:31.961+0000 7f6d0b038780 -1 Falling back to public interface 2026-02-17T18:45:32.317 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:32.316+0000 7f6d0b038780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:45:32.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:32.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:32.922 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:45:32.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:32.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:32.923 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:33.020 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2974409277,v1:127.0.0.1:6801/2974409277] [v2:127.0.0.1:6802/2974409277,v1:127.0.0.1:6803/2974409277] exists,up acdbe1ca-ddcb-4aa0-a6a7-eafea3bad582 2026-02-17T18:45:33.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:33.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:33.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:33.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:368: test_kill_daemons: kill_daemons td/ceph-helpers 0 osd 1 2026-02-17T18:45:33.021 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:33.021 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:33.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:33.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:33.022 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:34.027 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 1 2026-02-17T18:45:34.028 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:372: test_kill_daemons: kill_daemons td/ceph-helpers TERM osd 2026-02-17T18:45:34.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:34.028 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:34.029 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:34.029 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:34.029 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:34.336 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:34.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:376: test_kill_daemons: kill_daemons td/ceph-helpers TERM mgr 2026-02-17T18:45:34.337 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:34.337 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:34.337 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:34.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:34.338 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:34.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:34.444 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:380: test_kill_daemons: kill_daemons td/ceph-helpers TERM 2026-02-17T18:45:34.444 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:34.445 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:34.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:34.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:34.445 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:34.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:34.753 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:381: test_kill_daemons: timeout 5 ceph status 2026-02-17T18:45:39.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:382: test_kill_daemons: teardown td/ceph-helpers 2026-02-17T18:45:39.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:39.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:39.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:39.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:39.758 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:39.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:39.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:39.759 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:39.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:39.763 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:39.764 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:39.764 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:39.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:39.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:39.765 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:39.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:39.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:39.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:39.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:39.767 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:39.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:39.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:39.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:39.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:39.771 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:39.771 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:39.773 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:39.773 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_objectstore_tool td/ceph-helpers 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: test_objectstore_tool: local dir=td/ceph-helpers 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: test_objectstore_tool: setup td/ceph-helpers 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:39.774 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:39.775 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:39.775 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:39.775 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:39.775 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:39.775 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:39.776 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:39.776 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:39.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:39.777 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:39.777 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:39.778 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:39.778 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:39.778 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:39.779 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:39.780 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:39.780 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:39.781 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:39.781 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:39.781 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:39.781 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:39.782 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:39.782 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:39.782 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:45:39.783 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:45:39.783 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:39.783 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:39.784 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:45:39.784 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1301: test_objectstore_tool: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:45:39.786 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:39.803 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:45:39.804 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:39.804 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:39.804 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:39.804 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:39.804 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:39.804 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:39.804 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:39.828 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:45:39.830 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:45:39.830 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:39.830 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:39.830 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:45:39.830 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:45:39.830 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:39.831 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:39.831 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:39.831 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:39.831 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:39.831 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:39.831 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:39.832 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:39.832 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:45:39.872 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:45:39.872 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:39.872 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:39.872 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:45:39.873 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:45:39.873 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:39.873 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:39.873 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:39.874 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:39.874 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:39.874 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:39.874 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:39.874 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:39.874 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: test_objectstore_tool: local osd=0 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: test_objectstore_tool: run_mgr td/ceph-helpers x 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:45:39.915 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:45:40.013 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:45:40.013 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:40.013 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:40.013 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:40.014 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:40.014 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:40.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:40.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:40.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:40.033 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: test_objectstore_tool: run_osd td/ceph-helpers 0 2026-02-17T18:45:40.033 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:45:40.033 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:45:40.033 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:45:40.033 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:45:40.033 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:45:40.034 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:45:40.035 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:45:40.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:45:40.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:45:40.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:45:40.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:45:40.036 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:45:40.036 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:45:40.037 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c8d19fff-0b75-4133-ac3d-ef803807af20 2026-02-17T18:45:40.037 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c8d19fff-0b75-4133-ac3d-ef803807af20' 2026-02-17T18:45:40.037 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 c8d19fff-0b75-4133-ac3d-ef803807af20 2026-02-17T18:45:40.037 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:45:40.043 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBUt5Rp2Q+IAhAAOY2AwdEihTWeUNLt9FHSpA== 2026-02-17T18:45:40.044 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBUt5Rp2Q+IAhAAOY2AwdEihTWeUNLt9FHSpA=="}' 2026-02-17T18:45:40.044 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c8d19fff-0b75-4133-ac3d-ef803807af20 -i td/ceph-helpers/0/new.json 2026-02-17T18:45:40.144 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:40.150 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:45:40.151 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBUt5Rp2Q+IAhAAOY2AwdEihTWeUNLt9FHSpA== --osd-uuid c8d19fff-0b75-4133-ac3d-ef803807af20 2026-02-17T18:45:40.166 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.165+0000 7fcb48a33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:40.168 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.167+0000 7fcb48a33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:40.169 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.168+0000 7fcb48a33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:40.169 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.168+0000 7fcb48a33780 -1 bdev(0x3b76c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:45:40.170 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.168+0000 7fcb48a33780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:45:40.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:45:40.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:45:40.553 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:45:40.553 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:45:40.554 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:45:40.656 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:45:40.656 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:45:40.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:40.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:45:40.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:45:40.657 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:45:40.671 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.670+0000 7fb2531d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:40.672 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.671+0000 7fb2531d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:40.673 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.672+0000 7fb2531d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:40.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:45:40.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:40.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:40.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:40.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:40.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:40.757 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:40.758 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:40.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:40.758 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:40.856 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:40.891 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:40.889+0000 7fb2531d6780 -1 Falling back to public interface 2026-02-17T18:45:41.219 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:41.217+0000 7fb2531d6780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:45:41.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:41.858 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:41.858 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:45:41.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:41.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:41.859 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:41.955 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/823695768,v1:127.0.0.1:6801/823695768] [v2:127.0.0.1:6802/823695768,v1:127.0.0.1:6803/823695768] exists,up c8d19fff-0b75-4133-ac3d-ef803807af20 2026-02-17T18:45:41.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:41.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:41.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:41.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: test_objectstore_tool: create_rbd_pool 2026-02-17T18:45:41.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:45:42.049 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:45:42.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:45:42.055 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:45:42.279 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:45:42.287 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:45:43.288 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:45:43.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: test_objectstore_tool: wait_for_clean 2026-02-17T18:45:43.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:45:43.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:45:43.569 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:45:43.569 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:45:43.570 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:43.570 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:43.570 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:43.570 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:43.570 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:43.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:45:43.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:45:43.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:45:43.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:45:43.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:45:43.652 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:45:43.876 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:45:43.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:45:43.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:45:43.877 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:45:43.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:45:43.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:45:43.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:45:43.940 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:45:43.941 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:43.941 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:45:43.941 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:45:43.942 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:45:43.942 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:45:43.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:45:43.943 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:45:43.943 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:45:43.944 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:44.171 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:45:44.172 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:45:44.172 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:44.172 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:44.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:45:44.487 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:45:44.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:45:44.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:45:44.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:45:44.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:45:44.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:45:44.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:45:44.710 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:45:44.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:45:44.710 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:44.711 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:45.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:45:45.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:45:45.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:45:45.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1307: test_objectstore_tool: rados --pool rbd put GROUP /etc/group 2026-02-17T18:45:45.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1308: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 GROUP get-bytes 2026-02-17T18:45:45.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-17T18:45:45.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-17T18:45:45.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-17T18:45:45.039 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 GROUP get-bytes 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1309: test_objectstore_tool: diff - /etc/group 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:45.040 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 GROUP get-bytes 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:45.346 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 GROUP get-bytes 2026-02-17T18:45:45.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/ceph-helpers 0 2026-02-17T18:45:45.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-17T18:45:45.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-17T18:45:45.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-17T18:45:45.432 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-17T18:45:45.433 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:45:45.434 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr:start osd.0 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:45.435 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-17T18:45:45.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-17T18:45:45.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-17T18:45:45.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-17T18:45:45.436 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-17T18:45:45.446 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:45.445+0000 7f0ed57c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:45.448 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:45.447+0000 7f0ed57c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:45.449 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:45.447+0000 7f0ed57c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:45.592 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:45.590+0000 7f0ed57c6780 -1 Falling back to public interface 2026-02-17T18:45:45.658 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:0 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:45.659 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:45.889 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:45.890 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:45.889+0000 7f0ed57c6780 -1 osd.0 12 log_to_monitors true 2026-02-17T18:45:46.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:46.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:46.890 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:46.890 INFO:tasks.workunit.client.0.trial068.stderr:1 2026-02-17T18:45:46.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:46.891 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:osd.0 up in weight 1 up_from 15 up_thru 15 down_at 13 last_clean_interval [5,12) [v2:127.0.0.1:6800/705744353,v1:127.0.0.1:6801/705744353] [v2:127.0.0.1:6802/705744353,v1:127.0.0.1:6803/705744353] exists,up c8d19fff-0b75-4133-ac3d-ef803807af20 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:45:47.122 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:45:47.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:45:47.123 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:47.123 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:47.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:47.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:47.123 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:47.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:45:47.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:45:47.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:45:47.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:45:47.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:45:47.203 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:45:47.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:45:47.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:45:47.430 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:45:47.430 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:45:47.495 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-17T18:45:47.495 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-17T18:45:47.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-64424509443' 2026-02-17T18:45:47.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:45:47.496 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-64424509443 2026-02-17T18:45:47.496 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:45:47.497 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:45:47.497 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-64424509443 2026-02-17T18:45:47.497 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:45:47.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509443 2026-02-17T18:45:47.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 64424509443' 2026-02-17T18:45:47.499 INFO:tasks.workunit.client.0.trial068.stderr:waiting osd.0 seq 64424509443 2026-02-17T18:45:47.499 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:47.728 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 64424509443 2026-02-17T18:45:47.728 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:48.729 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:45:48.729 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:48.956 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-17T18:45:48.956 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:45:48.957 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:48.957 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:49.268 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:45:49.269 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:45:49.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:45:49.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:45:49.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:45:49.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:45:49.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:45:49.269 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:45:49.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:45:49.498 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:45:49.498 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:49.498 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:49.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:45:49.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:45:49.815 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1310: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-17T18:45:49.816 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-17T18:45:49.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-17T18:45:49.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-17T18:45:49.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:49.817 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:49.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:49.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:49.817 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-17T18:45:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-17T18:45:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-17T18:45:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-17T18:45:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-17T18:45:49.921 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:49.922 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 NOTEXISTS get-bytes 2026-02-17T18:45:49.972 INFO:tasks.workunit.client.0.trial068.stderr:No object id 'NOTEXISTS' found or invalid JSON specified 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: return 1 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: return 1 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1311: test_objectstore_tool: teardown td/ceph-helpers 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:50.014 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:50.120 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:50.121 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:50.121 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:50.122 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:50.123 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:50.123 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:50.124 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:50.125 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:50.125 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:50.126 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:50.127 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:50.127 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:50.131 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:50.131 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.131 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.131 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_schedule_scrub td/ceph-helpers 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1987: test_pg_schedule_scrub: local dir=td/ceph-helpers 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1989: test_pg_schedule_scrub: setup td/ceph-helpers 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:45:50.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:45:50.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:45:50.135 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:45:50.135 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:45:50.136 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:45:50.136 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:45:50.137 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:45:50.137 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:45:50.137 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:45:50.138 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:50.138 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:45:50.138 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:45:50.138 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:45:50.139 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:45:50.140 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:45:50.140 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:45:50.141 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:45:50.141 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.142 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:45:50.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:45:50.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:45:50.142 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:45:50.143 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:45:50.144 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.144 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.144 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:45:50.144 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:45:50.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:45:50.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:45:50.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:45:50.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1990: test_pg_schedule_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:50.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:45:50.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:45:50.146 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:45:50.147 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:45:50.147 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:45:50.147 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:50.163 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:45:50.164 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:50.164 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:50.164 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:50.164 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.164 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.164 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:50.164 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:45:50.189 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:45:50.189 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:45:50.190 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:50.190 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:50.190 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:45:50.190 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:50.191 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:45:50.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:45:50.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:45:50.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:45:50.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:45:50.233 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:45:50.234 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:45:50.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1991: test_pg_schedule_scrub: run_mgr td/ceph-helpers x --mgr_stats_period=1 2026-02-17T18:45:50.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:45:50.278 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:45:50.279 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:45:50.279 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:45:50.279 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:45:50.279 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:45:50.377 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:45:50.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:50.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:50.378 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:50.378 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.378 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:50.378 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:45:50.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T18:45:50.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1992: test_pg_schedule_scrub: run_osd td/ceph-helpers 0 2026-02-17T18:45:50.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:45:50.395 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:45:50.396 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:45:50.397 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:45:50.398 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=000c8281-fbee-4f97-8ed6-9b2b830a482f 2026-02-17T18:45:50.399 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 000c8281-fbee-4f97-8ed6-9b2b830a482f 2026-02-17T18:45:50.399 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 000c8281-fbee-4f97-8ed6-9b2b830a482f' 2026-02-17T18:45:50.399 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:45:50.409 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBet5RpyLdVGBAAwNfRjgRdO9OPOEChMl5L6g== 2026-02-17T18:45:50.409 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBet5RpyLdVGBAAwNfRjgRdO9OPOEChMl5L6g=="}' 2026-02-17T18:45:50.409 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 000c8281-fbee-4f97-8ed6-9b2b830a482f -i td/ceph-helpers/0/new.json 2026-02-17T18:45:50.515 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:50.522 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:45:50.523 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBet5RpyLdVGBAAwNfRjgRdO9OPOEChMl5L6g== --osd-uuid 000c8281-fbee-4f97-8ed6-9b2b830a482f 2026-02-17T18:45:50.539 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:50.538+0000 7ff27ea31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:50.540 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:50.539+0000 7ff27ea31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:50.541 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:50.540+0000 7ff27ea31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:50.541 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:50.540+0000 7ff27ea31780 -1 bdev(0x2cd8c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:45:50.542 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:50.541+0000 7ff27ea31780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:45:50.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:45:50.957 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:45:50.958 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:45:50.958 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:45:50.959 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:45:51.059 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:45:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:45:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:45:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:45:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:45:51.060 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:45:51.073 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:51.072+0000 7faef3e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:51.075 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:51.074+0000 7faef3e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:51.076 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:51.075+0000 7faef3e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:45:51.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:45:51.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:45:51.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:45:51.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:45:51.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:45:51.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:51.157 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:45:51.158 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:45:51.158 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:51.158 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:51.249 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:51.247+0000 7faef3e2c780 -1 Falling back to public interface 2026-02-17T18:45:51.255 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:45:51.571 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:45:51.570+0000 7faef3e2c780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:45:52.256 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:45:52.256 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:45:52.257 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:45:52.257 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:45:52.258 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:45:52.258 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:45:52.353 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1302261039,v1:127.0.0.1:6801/1302261039] [v2:127.0.0.1:6802/1302261039,v1:127.0.0.1:6803/1302261039] exists,up 000c8281-fbee-4f97-8ed6-9b2b830a482f 2026-02-17T18:45:52.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:45:52.354 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:45:52.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:45:52.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1993: test_pg_schedule_scrub: create_rbd_pool 2026-02-17T18:45:52.355 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:45:52.446 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:45:52.451 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:45:52.452 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:45:52.724 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:45:52.731 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:45:53.733 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:45:54.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1994: test_pg_schedule_scrub: wait_for_clean 2026-02-17T18:45:54.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:45:54.012 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:45:54.013 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:45:54.013 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:45:54.013 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:54.013 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:54.013 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:54.013 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:54.013 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:54.094 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:45:54.094 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:45:54.094 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:45:54.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:45:54.095 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:45:54.095 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:45:54.324 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:45:54.325 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:45:54.325 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:45:54.325 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:45:54.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-17T18:45:54.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-17T18:45:54.391 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T18:45:54.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:45:54.392 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-17T18:45:54.392 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:45:54.392 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:45:54.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-17T18:45:54.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:45:54.394 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-17T18:45:54.394 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836484 2026-02-17T18:45:54.394 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T18:45:54.394 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:54.619 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T18:45:54.620 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:55.621 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:45:55.621 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:55.846 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T18:45:55.847 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:56.847 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:45:56.848 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:57.083 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836484 2026-02-17T18:45:57.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:45:57.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:57.084 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:45:57.393 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:45:57.615 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:45:57.615 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:45:57.615 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:45:57.616 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:45:57.930 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1995: test_pg_schedule_scrub: pg_schedule_scrub 1.0 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1969: pg_schedule_scrub: local pgid=1.0 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: pg_schedule_scrub: wait_for_pg_clean 1.0 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T18:45:57.931 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:45:57.932 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:45:57.932 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:45:57.932 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:45:57.932 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:45:58.133 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T18:45:58.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-17T18:45:58.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-17T18:45:58.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-17T18:45:58.134 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:45:58.134 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:45:58.358 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:45:58.358 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:45:58.359 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:45:58.359 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:45:58.423 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-17T18:45:58.424 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-17T18:45:58.424 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-17T18:45:58.424 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:45:58.424 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-17T18:45:58.424 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:45:58.424 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:45:58.425 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-17T18:45:58.425 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:45:58.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-17T18:45:58.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-17T18:45:58.427 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836490 2026-02-17T18:45:58.427 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:58.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-17T18:45:58.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:45:59.654 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:45:59.654 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:45:59.879 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-17T18:45:59.879 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:46:00.879 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:46:00.880 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:46:01.107 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836492 -lt 21474836490 2026-02-17T18:46:01.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T18:46:01.108 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T18:46:01.108 INFO:tasks.workunit.client.0.trial068.stdout:#---------- 1.0 loop 0 2026-02-17T18:46:01.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T18:46:01.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T18:46:01.109 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T18:46:01.109 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T18:46:01.109 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T18:46:01.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-17T18:46:01.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T18:46:01.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-17T18:46:01.175 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-17T18:46:01.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: get_last_scrub_stamp 1.0 2026-02-17T18:46:01.175 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:01.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:01.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:01.176 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:01.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: local last_scrub=2026-02-17T18:45:52.662240+0000 2026-02-17T18:46:01.396 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: pg_schedule_scrub: ceph tell 1.0 schedule-scrub 2026-02-17T18:46:01.459 INFO:tasks.workunit.client.0.trial068.stdout:{ 2026-02-17T18:46:01.459 INFO:tasks.workunit.client.0.trial068.stdout: "deep": false, 2026-02-17T18:46:01.459 INFO:tasks.workunit.client.0.trial068.stdout: "must": false, 2026-02-17T18:46:01.459 INFO:tasks.workunit.client.0.trial068.stdout: "stamp": "2026-02-15T06:46:01.459182+0000" 2026-02-17T18:46:01.459 INFO:tasks.workunit.client.0.trial068.stdout:} 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: pg_schedule_scrub: wait_for_scrub 1.0 2026-02-17T18:45:52.662240+0000 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-17T18:45:52.662240+0000 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:01.464 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:01.465 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:01.465 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:01.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T18:45:52.662240+0000 '>' 2026-02-17T18:45:52.662240+0000 2026-02-17T18:46:01.691 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T18:46:02.692 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T18:46:02.693 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:02.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:02.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:02.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:02.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:02.694 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:02.928 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-15T06:46:01.459182+0000 '>' 2026-02-17T18:45:52.662240+0000 2026-02-17T18:46:02.928 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T18:46:03.928 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T18:46:03.928 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:03.929 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:03.929 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:03.929 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:03.929 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:03.929 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:04.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-15T06:46:01.459182+0000 '>' 2026-02-17T18:45:52.662240+0000 2026-02-17T18:46:04.152 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T18:46:05.154 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T18:46:05.154 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:05.154 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:05.154 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:05.155 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:05.155 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:05.155 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:05.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T18:46:02.445012+0000 '>' 2026-02-17T18:45:52.662240+0000 2026-02-17T18:46:05.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-17T18:46:05.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1996: test_pg_schedule_scrub: teardown td/ceph-helpers 2026-02-17T18:46:05.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:46:05.379 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:46:05.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:46:05.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:46:05.380 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:46:05.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:46:05.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:46:05.380 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:46:05.485 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:46:05.486 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:46:05.486 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:46:05.487 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:46:05.488 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:46:05.488 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:46:05.488 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:46:05.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:46:05.489 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:46:05.490 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:46:05.490 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:46:05.491 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:46:05.492 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:46:05.492 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:46:05.496 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:46:05.496 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.496 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.496 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:46:05.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:46:05.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:46:05.498 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_scrub td/ceph-helpers 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: test_pg_scrub: local dir=td/ceph-helpers 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: test_pg_scrub: setup td/ceph-helpers 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:46:05.499 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:46:05.500 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:46:05.500 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:46:05.500 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:46:05.501 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T18:46:05.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T18:46:05.501 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T18:46:05.501 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T18:46:05.502 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T18:46:05.502 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T18:46:05.502 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:46:05.502 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T18:46:05.503 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T18:46:05.503 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T18:46:05.503 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T18:46:05.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T18:46:05.505 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T18:46:05.506 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T18:46:05.506 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.506 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.506 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T18:46:05.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T18:46:05.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T18:46:05.507 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-17T18:46:05.508 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-17T18:46:05.508 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.508 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.508 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.59203 2026-02-17T18:46:05.509 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-17T18:46:05.510 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-17T18:46:05.510 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-17T18:46:05.510 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-17T18:46:05.510 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: test_pg_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:46:05.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-17T18:46:05.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-17T18:46:05.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-17T18:46:05.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-17T18:46:05.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-17T18:46:05.511 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:46:05.529 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-17T18:46:05.529 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:46:05.529 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:46:05.529 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:46:05.529 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.529 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.529 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:46:05.530 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T18:46:05.552 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-17T18:46:05.553 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-17T18:46:05.554 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:46:05.554 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:46:05.554 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-17T18:46:05.554 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-17T18:46:05.555 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:46:05.555 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:46:05.555 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:46:05.555 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:46:05.555 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.556 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.556 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:46:05.556 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:46:05.556 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get fsid 2026-02-17T18:46:05.597 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-17T18:46:05.597 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-17T18:46:05.597 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-17T18:46:05.597 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-17T18:46:05.597 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-17T18:46:05.597 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-17T18:46:05.598 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-17T18:46:05.598 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-17T18:46:05.599 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-17T18:46:05.599 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.599 INFO:tasks.workunit.client.0.trial068.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.599 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.59203/ceph-mon.a.asok 2026-02-17T18:46:05.599 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-17T18:46:05.599 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.59203/ceph-mon.a.asok config get mon_host 2026-02-17T18:46:05.643 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: test_pg_scrub: run_mgr td/ceph-helpers x 2026-02-17T18:46:05.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-17T18:46:05.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-17T18:46:05.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-17T18:46:05.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-17T18:46:05.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-17T18:46:05.644 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T18:46:05.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-17T18:46:05.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:46:05.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:46:05.745 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:46:05.745 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.746 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:46:05.746 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:46:05.746 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T18:46:05.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: test_pg_scrub: run_osd td/ceph-helpers 0 2026-02-17T18:46:05.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-17T18:46:05.765 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-17T18:46:05.766 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' 2026-02-17T18:46:05.767 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-17T18:46:05.768 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-17T18:46:05.769 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-17T18:46:05.769 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=451ec615-255d-46a3-8bf4-048d239007b3 2026-02-17T18:46:05.769 INFO:tasks.workunit.client.0.trial068.stdout:add osd0 451ec615-255d-46a3-8bf4-048d239007b3 2026-02-17T18:46:05.770 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 451ec615-255d-46a3-8bf4-048d239007b3' 2026-02-17T18:46:05.770 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-17T18:46:05.775 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBtt5Rp0LAiLhAAXRMdF1cDAs7WrPPYmFGOjQ== 2026-02-17T18:46:05.775 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBtt5Rp0LAiLhAAXRMdF1cDAs7WrPPYmFGOjQ=="}' 2026-02-17T18:46:05.775 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 451ec615-255d-46a3-8bf4-048d239007b3 -i td/ceph-helpers/0/new.json 2026-02-17T18:46:05.878 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:46:05.884 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-17T18:46:05.885 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBtt5Rp0LAiLhAAXRMdF1cDAs7WrPPYmFGOjQ== --osd-uuid 451ec615-255d-46a3-8bf4-048d239007b3 2026-02-17T18:46:05.901 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:05.900+0000 7fa23a637780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:46:05.902 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:05.901+0000 7fa23a637780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:46:05.903 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:05.902+0000 7fa23a637780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:46:05.904 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:05.902+0000 7fa23a637780 -1 bdev(0x246cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-17T18:46:05.904 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:05.902+0000 7fa23a637780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-17T18:46:06.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-17T18:46:06.313 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-17T18:46:06.314 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-17T18:46:06.314 INFO:tasks.workunit.client.0.trial068.stdout:adding osd0 key to auth repository 2026-02-17T18:46:06.315 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T18:46:06.416 INFO:tasks.workunit.client.0.trial068.stdout:start osd.0 2026-02-17T18:46:06.417 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-17T18:46:06.417 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=6ea53734-b674-4188-bdec-9c64ebdd92c0 --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.59203/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T18:46:06.417 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-17T18:46:06.417 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-17T18:46:06.418 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-17T18:46:06.432 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:06.430+0000 7f1cefe2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:46:06.433 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:06.432+0000 7f1cefe2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:46:06.434 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:06.433+0000 7f1cefe2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T18:46:06.514 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-17T18:46:06.515 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-17T18:46:06.515 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-17T18:46:06.515 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-17T18:46:06.515 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-17T18:46:06.515 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:46:06.515 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-17T18:46:06.515 INFO:tasks.workunit.client.0.trial068.stdout:0 2026-02-17T18:46:06.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:46:06.516 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:46:06.612 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-17T18:46:06.636 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:06.635+0000 7f1cefe2d780 -1 Falling back to public interface 2026-02-17T18:46:07.032 INFO:tasks.workunit.client.0.trial068.stderr:2026-02-17T18:46:07.031+0000 7f1cefe2d780 -1 osd.0 0 log_to_monitors true 2026-02-17T18:46:07.613 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-17T18:46:07.614 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-17T18:46:07.614 INFO:tasks.workunit.client.0.trial068.stdout:1 2026-02-17T18:46:07.615 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-17T18:46:07.615 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-17T18:46:07.615 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-17T18:46:07.712 INFO:tasks.workunit.client.0.trial068.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4194980990,v1:127.0.0.1:6801/4194980990] [v2:127.0.0.1:6802/4194980990,v1:127.0.0.1:6803/4194980990] exists,up 451ec615-255d-46a3-8bf4-048d239007b3 2026-02-17T18:46:07.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-17T18:46:07.712 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-17T18:46:07.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-17T18:46:07.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1947: test_pg_scrub: create_rbd_pool 2026-02-17T18:46:07.713 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T18:46:07.806 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' does not exist 2026-02-17T18:46:07.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-17T18:46:07.812 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-17T18:46:08.085 INFO:tasks.workunit.client.0.trial068.stderr:pool 'rbd' already exists 2026-02-17T18:46:08.092 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-17T18:46:09.093 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-17T18:46:09.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1948: test_pg_scrub: wait_for_clean 2026-02-17T18:46:09.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-17T18:46:09.374 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-17T18:46:09.375 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-17T18:46:09.375 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T18:46:09.375 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:46:09.375 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:46:09.375 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:46:09.375 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:46:09.375 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:46:09.455 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T18:46:09.455 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-17T18:46:09.455 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-17T18:46:09.455 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-17T18:46:09.455 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:46:09.455 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:46:09.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:46:09.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:46:09.677 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:46:09.678 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:46:09.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-17T18:46:09.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-17T18:46:09.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-17T18:46:09.742 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:46:09.742 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-17T18:46:09.742 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:46:09.743 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:46:09.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-17T18:46:09.743 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:46:09.744 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-17T18:46:09.744 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-17T18:46:09.745 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836483 2026-02-17T18:46:09.745 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:46:09.974 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:46:09.974 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:46:10.975 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:46:10.975 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:46:11.199 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-17T18:46:11.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:46:12.200 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:46:12.201 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:46:12.437 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-17T18:46:12.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-17T18:46:12.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:46:12.438 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:46:12.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-17T18:46:12.749 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-17T18:46:12.749 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-17T18:46:12.749 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-17T18:46:12.749 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T18:46:12.750 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T18:46:12.750 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T18:46:12.750 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T18:46:12.988 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-17T18:46:12.988 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T18:46:12.989 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-17T18:46:12.989 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1949: test_pg_scrub: pg_scrub 1.0 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-17T18:46:13.302 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T18:46:13.303 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:46:13.303 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:46:13.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:46:13.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:46:13.303 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:46:13.503 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T18:46:13.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-17T18:46:13.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-17T18:46:13.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-17T18:46:13.504 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-17T18:46:13.504 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:46:13.730 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:46:13.730 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:46:13.730 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:46:13.730 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-17T18:46:13.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-17T18:46:13.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-17T18:46:13.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T18:46:13.794 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-17T18:46:13.794 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836486 2026-02-17T18:46:13.794 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-17T18:46:13.795 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-17T18:46:13.795 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-17T18:46:13.796 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-17T18:46:13.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836486 2026-02-17T18:46:13.797 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T18:46:13.797 INFO:tasks.workunit.client.0.trial068.stdout:waiting osd.0 seq 21474836486 2026-02-17T18:46:13.797 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:46:14.020 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-17T18:46:14.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:46:15.021 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T18:46:15.022 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:46:15.248 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-17T18:46:15.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-17T18:46:16.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T18:46:16.250 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T18:46:16.474 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T18:46:16.475 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T18:46:16.475 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T18:46:16.475 INFO:tasks.workunit.client.0.trial068.stdout:#---------- 1.0 loop 0 2026-02-17T18:46:16.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T18:46:16.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T18:46:16.476 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T18:46:16.476 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T18:46:16.476 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T18:46:16.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-17T18:46:16.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T18:46:16.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-17T18:46:16.541 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-17T18:46:16.541 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T18:46:16.541 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:16.542 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:16.542 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:16.542 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:16.762 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-17T18:46:08.021113+0000 2026-02-17T18:46:16.763 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-17T18:46:16.974 INFO:tasks.workunit.client.0.trial068.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-17T18:46:16.982 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-17T18:46:08.021113+0000 2026-02-17T18:46:16.982 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-17T18:46:16.982 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-17T18:46:08.021113+0000 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:16.983 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:17.202 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T18:46:08.021113+0000 '>' 2026-02-17T18:46:08.021113+0000 2026-02-17T18:46:17.202 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T18:46:18.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T18:46:18.203 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:18.204 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:18.204 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:18.204 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:18.204 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:18.205 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:18.425 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T18:46:08.021113+0000 '>' 2026-02-17T18:46:08.021113+0000 2026-02-17T18:46:18.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T18:46:19.426 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T18:46:19.427 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:19.427 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:19.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:19.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:19.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:19.428 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:19.651 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T18:46:08.021113+0000 '>' 2026-02-17T18:46:08.021113+0000 2026-02-17T18:46:19.651 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-17T18:46:20.652 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-17T18:46:20.653 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-17T18:46:20.653 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T18:46:20.654 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-17T18:46:20.654 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T18:46:20.654 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T18:46:20.654 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T18:46:20.876 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-17T18:46:17.083164+0000 '>' 2026-02-17T18:46:08.021113+0000 2026-02-17T18:46:20.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-17T18:46:20.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1950: test_pg_scrub: kill_daemons td/ceph-helpers KILL osd 2026-02-17T18:46:20.877 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T18:46:20.877 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T18:46:20.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T18:46:20.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T18:46:20.877 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T18:46:20.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T18:46:20.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: WAIT_FOR_CLEAN_TIMEOUT=10 2026-02-17T18:46:20.983 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: TIMEOUT=2 2026-02-17T18:46:20.984 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: pg_scrub 1.0 2026-02-17T18:46:20.984 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-17T18:46:20.984 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T18:46:20.984 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-17T18:46:20.984 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 10 1 3 2026-02-17T18:46:20.985 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-17T18:46:20.985 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-17T18:46:20.985 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-17T18:46:20.985 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-17T18:46:20.985 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-17T18:46:21.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '1') 2026-02-17T18:46:21.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-17T18:46:21.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-17T18:46:21.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-17T18:46:21.017 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=2 2026-02-17T18:46:21.017 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-17T18:46:21.243 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-17T18:46:21.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-17T18:46:21.244 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-17T18:46:21.244 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 2 ceph tell osd.0 flush_pg_stats 2026-02-17T18:46:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-17T18:46:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-17T18:46:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-17T18:46:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-17T18:46:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T18:46:23.249 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T18:46:23.250 INFO:tasks.workunit.client.0.trial068.stdout:#---------- 1.0 loop 0 2026-02-17T18:46:23.250 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T18:46:23.250 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T18:46:23.251 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T18:46:23.251 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T18:46:23.251 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T21:43:27.207 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1: is_pg_clean: teardown td/ceph-helpers 1 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=1 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-17T21:43:27.237 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-17T21:43:27.238 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-17T21:43:27.238 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-17T21:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-17T21:43:27.316 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-17T21:43:27.318 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:43:27.318 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-17T21:43:27.320 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:43:27.320 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-17T21:43:27.320 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:43:27.322 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:43:27.322 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-17T21:43:27.322 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-17T21:43:27.322 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:43:27.323 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:43:27.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-17T21:43:27.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:181: teardown: '[' -n '' ']' 2026-02-17T21:43:27.324 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:185: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-17T21:43:27.325 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:186: teardown: mv td/ceph-helpers/mgr.x.log td/ceph-helpers/mon.a.log td/ceph-helpers/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-17T21:43:27.326 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-17T21:43:27.329 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-17T21:43:27.329 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-17T21:43:27.329 INFO:tasks.workunit.client.0.trial068.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.59203 2026-02-17T21:43:27.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.59203 2026-02-17T21:43:27.330 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-17T21:43:27.331 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-17T21:43:27.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T21:43:27.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 0 >= 5 )) 2026-02-17T21:43:27.331 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 1 2026-02-17T21:43:28.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-17T21:43:28.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-17T21:43:28.332 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 1' 2026-02-17T21:43:28.332 INFO:tasks.workunit.client.0.trial068.stdout:#---------- 1.0 loop 1 2026-02-17T21:43:28.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T21:43:28.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-17T21:43:28.333 INFO:tasks.workunit.client.0.trial068.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-17T21:43:28.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-17T21:43:28.333 INFO:tasks.workunit.client.0.trial068.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-17T21:44:27.237 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-17T21:45:27.237 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-17T21:45:27.238 INFO:tasks.workunit:Stopping ['misc'] on client.0... 2026-02-17T21:45:27.238 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-17T21:45:27.569 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/remote.py", line 575, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test misc/test-ceph-helpers.sh) on trial068 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh' 2026-02-17T21:45:27.575 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T21:45:27.590 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T21:45:27.591 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T21:45:27.722 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T21:45:27.733 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T21:45:27.733 DEBUG:teuthology.orchestra.run.trial068:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T21:45:27.750 INFO:teuthology.orchestra.run.trial068.stderr:bash: line 1: ntpq: command not found 2026-02-17T21:45:27.786 INFO:teuthology.orchestra.run.trial068.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T21:45:27.786 INFO:teuthology.orchestra.run.trial068.stdout:=============================================================================== 2026-02-17T21:45:27.786 INFO:teuthology.orchestra.run.trial068.stdout:^- ntp.maxhost.io 2 9 377 471 +522us[ +378us] +/- 55ms 2026-02-17T21:45:27.786 INFO:teuthology.orchestra.run.trial068.stdout:^* ntp.netlinkify.com 2 10 377 289 -13us[ -157us] +/- 8018us 2026-02-17T21:45:27.786 INFO:teuthology.orchestra.run.trial068.stdout:^- anaheim.capsaic.in 2 10 377 11 +577us[ +577us] +/- 40ms 2026-02-17T21:45:27.786 INFO:teuthology.orchestra.run.trial068.stdout:^- 143-42-229-154.ip.linode> 2 7 377 72 -739us[ -739us] +/- 39ms 2026-02-17T21:45:27.787 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T21:45:27.798 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T21:45:27.798 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T21:45:27.812 DEBUG:teuthology.orchestra.run.trial068:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T21:45:27.867 DEBUG:teuthology.orchestra.run.trial068:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-17T21:45:27.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:45:27.934 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T21:45:27.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T21:45:27.963 INFO:teuthology.task.internal:Duration was 11470.059377 seconds 2026-02-17T21:45:27.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T21:45:27.978 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T21:45:27.978 DEBUG:teuthology.orchestra.run.trial068:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T21:45:28.014 INFO:teuthology.orchestra.run.trial068.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T21:45:28.145 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T21:45:28.145 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial068.front.sepia.ceph.com 2026-02-17T21:45:28.146 DEBUG:teuthology.orchestra.run.trial068:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T21:45:28.206 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T21:45:28.206 DEBUG:teuthology.orchestra.run.trial068:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T21:45:28.290 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T21:45:28.291 DEBUG:teuthology.orchestra.run.trial068:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T21:45:28.313 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T21:45:28.314 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T21:45:28.314 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T21:45:28.315 INFO:teuthology.orchestra.run.trial068.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T21:45:28.315 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T21:45:28.318 INFO:teuthology.orchestra.run.trial068.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 90.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T21:45:28.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T21:45:28.336 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T21:45:28.336 DEBUG:teuthology.orchestra.run.trial068:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T21:45:28.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T21:45:28.400 DEBUG:teuthology.orchestra.run.trial068:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:45:28.448 INFO:teuthology.orchestra.run.trial068.stdout:kernel.core_pattern = core 2026-02-17T21:45:28.461 DEBUG:teuthology.orchestra.run.trial068:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:45:28.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:45:28.516 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T21:45:28.533 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T21:45:28.536 DEBUG:teuthology.misc:Transferring archived files from trial068:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_15:45:02-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/53767/remote/trial068 2026-02-17T21:45:28.538 DEBUG:teuthology.orchestra.run.trial068:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T21:45:42.308 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T21:45:42.308 DEBUG:teuthology.orchestra.run.trial068:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T21:45:42.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T21:45:42.406 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T21:45:42.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T21:45:42.420 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T21:45:42.420 DEBUG:teuthology.orchestra.run.trial068:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T21:45:42.437 INFO:teuthology.orchestra.run.trial068.stdout: 786511 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 21:45 /home/ubuntu/cephtest 2026-02-17T21:45:42.437 INFO:teuthology.orchestra.run.trial068.stdout: 786881 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 18:36 /home/ubuntu/cephtest/mnt.0 2026-02-17T21:45:42.437 INFO:teuthology.orchestra.run.trial068.stdout: 805111 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 18:43 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T21:45:42.437 INFO:teuthology.orchestra.run.trial068.stdout: 1064389 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 18:43 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T21:45:42.437 INFO:teuthology.orchestra.run.trial068.stdout: 1064390 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 21:43 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-17T21:45:42.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:45:42.438 INFO:teuthology.orchestra.run.trial068.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-17T21:45:42.439 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial068 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-17T21:45:42.440 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T21:45:42.452 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T21:45:42.467 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial068 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-17T21:45:42.470 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/misc} duration: 11470.059376955032 failure_reason: 'Command failed (workunit test misc/test-ceph-helpers.sh) on trial068 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-17T21:45:42.471 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T21:45:42.689 INFO:teuthology.run:FAIL