2026-02-13T19:20:02.843 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-13T19:20:02.858 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T19:20:02.933 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49813 branch: wip-yuri4-testing-2026-02-11-2126 description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/misc} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '49813' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri4-testing-2026-02-11-2126 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 workunit: branch: wip-yuri4-testing-2026-02-11-2126 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 5319 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2026-02-11-2126 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 targets: trial162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5CSM+dsvCUq03HO//sGivlOsngKqchrZsnXqHxM4RqqQC1QOIVOctKWgqqq+MhireDCFWeq1Pm595h/kD04Rw= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-13_16:37:03 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 2026-02-13T19:20:02.933 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa; will attempt to use it 2026-02-13T19:20:02.934 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks 2026-02-13T19:20:02.934 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-13T19:20:02.936 INFO:teuthology.task.internal:Checking packages... 2026-02-13T19:20:02.936 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'afefaa40503a4bf024e96f97dd7f852b75f00ad6' 2026-02-13T19:20:02.936 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T19:20:02.936 INFO:teuthology.packaging:ref: None 2026-02-13T19:20:02.936 INFO:teuthology.packaging:tag: None 2026-02-13T19:20:02.936 INFO:teuthology.packaging:branch: wip-yuri4-testing-2026-02-11-2126 2026-02-13T19:20:02.936 INFO:teuthology.packaging:sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:20:02.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri4-testing-2026-02-11-2126 2026-02-13T19:20:03.055 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5230.gafefaa40 2026-02-13T19:20:03.056 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-13T19:20:03.067 INFO:teuthology.task.internal:no buildpackages task found 2026-02-13T19:20:03.067 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-13T19:20:03.082 INFO:teuthology.task.internal:Saving configuration 2026-02-13T19:20:03.091 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-13T19:20:03.103 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-13T19:20:03.180 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial162.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49813', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 19:16:28.255203', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5CSM+dsvCUq03HO//sGivlOsngKqchrZsnXqHxM4RqqQC1QOIVOctKWgqqq+MhireDCFWeq1Pm595h/kD04Rw='} 2026-02-13T19:20:03.181 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-13T19:20:03.192 INFO:teuthology.task.internal:roles: ubuntu@trial162.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-13T19:20:03.192 INFO:teuthology.run_tasks:Running task console_log... 2026-02-13T19:20:03.278 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x782d669cd6c0>, signals=[15]) 2026-02-13T19:20:03.278 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-13T19:20:03.290 INFO:teuthology.task.internal:Opening connections... 2026-02-13T19:20:03.290 DEBUG:teuthology.task.internal:connecting to ubuntu@trial162.front.sepia.ceph.com 2026-02-13T19:20:03.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T19:20:03.354 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-13T19:20:03.366 DEBUG:teuthology.orchestra.run.trial162:> uname -m 2026-02-13T19:20:03.433 INFO:teuthology.orchestra.run.trial162.stdout:x86_64 2026-02-13T19:20:03.433 DEBUG:teuthology.orchestra.run.trial162:> cat /etc/os-release 2026-02-13T19:20:03.489 INFO:teuthology.orchestra.run.trial162.stdout:NAME="CentOS Stream" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:VERSION="9" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:ID="centos" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:ID_LIKE="rhel fedora" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:VERSION_ID="9" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:PLATFORM_ID="platform:el9" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:ANSI_COLOR="0;31" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:LOGO="fedora-logo-icon" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:HOME_URL="https://centos.org/" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T19:20:03.490 INFO:teuthology.orchestra.run.trial162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T19:20:03.491 INFO:teuthology.lock.ops:Updating trial162.front.sepia.ceph.com on lock server 2026-02-13T19:20:03.577 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-13T19:20:03.601 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-13T19:20:03.616 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-13T19:20:03.616 DEBUG:teuthology.orchestra.run.trial162:> test '!' -e /home/ubuntu/cephtest 2026-02-13T19:20:03.631 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-13T19:20:03.646 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-13T19:20:03.646 DEBUG:teuthology.orchestra.run.trial162:> test -z $(ls -A /var/lib/ceph) 2026-02-13T19:20:03.688 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-13T19:20:03.791 INFO:teuthology.run_tasks:Running task kernel... 2026-02-13T19:20:03.811 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-13T19:20:03.811 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-13T19:20:03.811 DEBUG:teuthology.orchestra.run.trial162:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T19:20:03.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T19:20:03.827 DEBUG:teuthology.orchestra.run.trial162:> uname -r 2026-02-13T19:20:03.882 INFO:teuthology.orchestra.run.trial162.stdout:5.14.0-645.el9.x86_64 2026-02-13T19:20:03.883 INFO:teuthology.task.kernel:Running kernel on trial162: 5.14.0-645.el9.x86_64 2026-02-13T19:20:03.883 DEBUG:teuthology.orchestra.run.trial162:> sudo yum install -y kernel 2026-02-13T19:20:04.943 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-13T19:20:08.106 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-13T19:20:11.384 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 7.8 MB 00:00 2026-02-13T19:20:12.213 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - Extras packages 180 kB/s | 20 kB 00:00 2026-02-13T19:20:12.478 INFO:teuthology.orchestra.run.trial162.stdout:Extra Packages for Enterprise Linux 106 MB/s | 20 MB 00:00 2026-02-13T19:20:15.708 INFO:teuthology.orchestra.run.trial162.stdout:lab-extras 638 kB/s | 50 kB 00:00 2026-02-13T19:20:16.767 INFO:teuthology.orchestra.run.trial162.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T19:20:16.788 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout: Package Architecture Version Repository Size 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T19:20:16.789 INFO:teuthology.orchestra.run.trial162.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout:Install 4 Packages 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 88 M 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 135 M 2026-02-13T19:20:16.790 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:20:17.004 INFO:teuthology.orchestra.run.trial162.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 5.9 MB/s | 768 kB 00:00 2026-02-13T19:20:17.154 INFO:teuthology.orchestra.run.trial162.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 61 MB/s | 17 MB 00:00 2026-02-13T19:20:17.550 INFO:teuthology.orchestra.run.trial162.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 56 MB/s | 31 MB 00:00 2026-02-13T19:20:17.604 INFO:teuthology.orchestra.run.trial162.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 54 MB/s | 40 MB 00:00 2026-02-13T19:20:17.604 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:20:17.604 INFO:teuthology.orchestra.run.trial162.stdout:Total 108 MB/s | 88 MB 00:00 2026-02-13T19:20:17.793 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:20:17.992 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:20:17.992 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:20:18.335 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:20:18.335 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:20:19.417 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:20:19.700 INFO:teuthology.orchestra.run.trial162.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T19:20:19.718 INFO:teuthology.orchestra.run.trial162.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T19:20:21.500 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T19:20:21.529 INFO:teuthology.orchestra.run.trial162.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T19:20:24.758 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T19:20:24.764 INFO:teuthology.orchestra.run.trial162.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T19:20:27.801 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T19:20:58.654 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T19:20:58.676 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T19:20:58.740 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T19:20:58.740 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T19:20:58.740 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T19:20:58.740 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T19:20:58.793 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T19:20:58.793 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:20:58.794 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:20:58.794 INFO:teuthology.orchestra.run.trial162.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T19:20:58.794 INFO:teuthology.orchestra.run.trial162.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T19:20:58.794 INFO:teuthology.orchestra.run.trial162.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T19:20:58.794 INFO:teuthology.orchestra.run.trial162.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T19:20:58.794 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:20:58.794 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:20:58.852 DEBUG:teuthology.orchestra.run.trial162:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T19:20:58.877 INFO:teuthology.orchestra.run.trial162.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T19:20:58.877 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T19:20:58.878 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T19:20:58.878 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T19:20:58.878 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T19:20:58.878 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-13T19:20:58.878 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial162.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T19:20:58.878 DEBUG:teuthology.orchestra.run.trial162:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T19:20:58.941 INFO:teuthology.orchestra.run.trial162.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T19:20:58.942 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T19:20:58.942 DEBUG:teuthology.orchestra.run.trial162:> sudo rpm -qi grub2-tools 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Name : grub2-tools 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Epoch : 1 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Version : 2.06 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Release : 118.el9 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Architecture: x86_64 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Group : Unspecified 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Size : 8393909 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:License : GPLv3+ 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T19:20:59.011 INFO:teuthology.orchestra.run.trial162.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:Packager : builder@centos.org 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:Vendor : CentOS 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:Summary : Support tools for GRUB. 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:Description : 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:hardware devices. 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:20:59.012 INFO:teuthology.orchestra.run.trial162.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T19:20:59.013 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T19:20:59.013 INFO:teuthology.task.kernel:Updating grub on trial162 to boot 5.14.0-677.el9.x86_64 2026-02-13T19:20:59.013 DEBUG:teuthology.orchestra.run.trial162:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T19:20:59.076 INFO:teuthology.orchestra.run.trial162.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T19:20:59.077 DEBUG:teuthology.orchestra.run.trial162:> sudo command -v grubby && echo yes || echo no 2026-02-13T19:20:59.140 INFO:teuthology.orchestra.run.trial162.stdout:/sbin/grubby 2026-02-13T19:20:59.142 INFO:teuthology.orchestra.run.trial162.stdout:yes 2026-02-13T19:20:59.142 DEBUG:teuthology.orchestra.run.trial162:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T19:20:59.206 INFO:teuthology.orchestra.run.trial162.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T19:20:59.207 DEBUG:teuthology.orchestra.run.trial162:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T19:20:59.281 DEBUG:teuthology.orchestra.run.trial162:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T19:20:59.495 INFO:teuthology.orchestra.run.trial162.stderr:Generating grub configuration file ... 2026-02-13T19:20:59.559 INFO:teuthology.orchestra.run.trial162.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T19:20:59.565 INFO:teuthology.orchestra.run.trial162.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T19:20:59.824 INFO:teuthology.orchestra.run.trial162.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T19:20:59.831 INFO:teuthology.orchestra.run.trial162.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T19:20:59.863 INFO:teuthology.orchestra.run.trial162.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T19:20:59.869 INFO:teuthology.orchestra.run.trial162.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T19:20:59.893 INFO:teuthology.orchestra.run.trial162.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T19:20:59.900 INFO:teuthology.orchestra.run.trial162.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T19:20:59.983 INFO:teuthology.orchestra.run.trial162.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T19:20:59.992 INFO:teuthology.orchestra.run.trial162.stderr:done 2026-02-13T19:20:59.993 DEBUG:teuthology.orchestra.run.trial162:> test -d /sys/firmware/efi 2026-02-13T19:21:00.009 DEBUG:teuthology.orchestra.run.trial162:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T19:21:00.076 INFO:teuthology.orchestra.run.trial162.stdout:/boot/efi/EFI/BOOT 2026-02-13T19:21:00.076 INFO:teuthology.orchestra.run.trial162.stdout:/boot/efi/EFI/centos 2026-02-13T19:21:00.077 DEBUG:teuthology.orchestra.run.trial162:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T19:21:00.142 INFO:teuthology.orchestra.run.trial162.stdout:no 2026-02-13T19:21:00.143 DEBUG:teuthology.orchestra.run.trial162:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T19:21:00.208 INFO:teuthology.orchestra.run.trial162.stdout:yes 2026-02-13T19:21:00.209 DEBUG:teuthology.orchestra.run.trial162:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T19:21:00.272 DEBUG:teuthology.orchestra.run.trial162:> sudo shutdown -r now 2026-02-13T19:21:00.347 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-13T19:21:30.346 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-13T19:21:30.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial162.front.sepia.ceph.com' 2026-02-13T19:21:30.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T19:22:30.363 DEBUG:teuthology.orchestra.remote:timed out 2026-02-13T19:22:39.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial162.front.sepia.ceph.com' 2026-02-13T19:22:39.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T19:22:39.481 DEBUG:teuthology.orchestra.run.trial162:> true 2026-02-13T19:22:39.638 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial162.front.sepia.ceph.com' 2026-02-13T19:22:39.638 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-13T19:22:39.638 DEBUG:teuthology.orchestra.run.trial162:> uname -r 2026-02-13T19:22:39.694 INFO:teuthology.orchestra.run.trial162.stdout:5.14.0-677.el9.x86_64 2026-02-13T19:22:39.695 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T19:22:39.695 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T19:22:39.695 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T19:22:40.696 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-13T19:22:40.697 DEBUG:teuthology.orchestra.run.trial162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-13T19:22:40.725 INFO:teuthology.orchestra.run.trial162.stdout:ttyS1 2026-02-13T19:22:40.744 DEBUG:teuthology.parallel:result is None 2026-02-13T19:22:40.744 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-13T19:22:40.756 INFO:teuthology.task.internal:Creating test directory... 2026-02-13T19:22:40.756 DEBUG:teuthology.orchestra.run.trial162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T19:22:40.800 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-13T19:22:40.812 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-13T19:22:40.826 INFO:teuthology.task.internal:Creating archive directory... 2026-02-13T19:22:40.826 DEBUG:teuthology.orchestra.run.trial162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T19:22:40.863 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-13T19:22:40.874 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-13T19:22:40.875 DEBUG:teuthology.orchestra.run.trial162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T19:22:40.924 INFO:teuthology.orchestra.run.trial162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:22:40.932 INFO:teuthology.orchestra.run.trial162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:22:40.934 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-13T19:22:40.949 INFO:teuthology.task.internal:Configuring sudo... 2026-02-13T19:22:40.949 DEBUG:teuthology.orchestra.run.trial162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T19:22:41.000 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-13T19:22:41.021 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-13T19:22:41.021 DEBUG:teuthology.orchestra.run.trial162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T19:22:41.056 DEBUG:teuthology.orchestra.run.trial162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T19:22:41.204 DEBUG:teuthology.orchestra.run.trial162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T19:22:41.228 DEBUG:teuthology.orchestra.run.trial162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T19:22:41.363 DEBUG:teuthology.orchestra.run.trial162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T19:22:41.389 DEBUG:teuthology.orchestra.run.trial162:> set -ex 2026-02-13T19:22:41.389 DEBUG:teuthology.orchestra.run.trial162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T19:22:41.453 DEBUG:teuthology.orchestra.run.trial162:> sudo service rsyslog restart 2026-02-13T19:22:41.520 INFO:teuthology.orchestra.run.trial162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T19:22:41.768 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-13T19:22:41.782 INFO:teuthology.task.internal:Starting timer... 2026-02-13T19:22:41.782 INFO:teuthology.run_tasks:Running task pcp... 2026-02-13T19:22:41.812 INFO:teuthology.run_tasks:Running task selinux... 2026-02-13T19:22:41.831 DEBUG:teuthology.orchestra.run.trial162:> sudo service auditd rotate 2026-02-13T19:22:41.862 INFO:teuthology.orchestra.run.trial162.stdout:Rotating logs: 2026-02-13T19:22:41.864 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-13T19:22:41.865 DEBUG:teuthology.orchestra.run.trial162:> /usr/sbin/getenforce 2026-02-13T19:22:41.921 INFO:teuthology.orchestra.run.trial162.stdout:Permissive 2026-02-13T19:22:41.921 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial162.front.sepia.ceph.com': 'permissive'} 2026-02-13T19:22:41.921 DEBUG:teuthology.orchestra.run.trial162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-13T19:22:41.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T19:22:41.988 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-13T19:22:41.988 DEBUG:teuthology.orchestra.run.trial162:> sudo /usr/sbin/setenforce permissive 2026-02-13T19:22:42.051 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-13T19:22:42.064 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-13T19:22:42.074 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-13T19:22:42.075 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-13T19:25:23.128 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial162.front.sepia.ceph.com')] 2026-02-13T19:25:23.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial162.front.sepia.ceph.com' 2026-02-13T19:25:23.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T19:25:23.241 DEBUG:teuthology.orchestra.run.trial162:> true 2026-02-13T19:25:23.324 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial162.front.sepia.ceph.com' 2026-02-13T19:25:23.324 INFO:teuthology.run_tasks:Running task clock... 2026-02-13T19:25:23.337 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-13T19:25:23.337 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T19:25:23.338 DEBUG:teuthology.orchestra.run.trial162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T19:25:23.391 INFO:teuthology.orchestra.run.trial162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T19:25:23.402 INFO:teuthology.orchestra.run.trial162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T19:25:23.422 INFO:teuthology.orchestra.run.trial162.stderr:sudo: ntpd: command not found 2026-02-13T19:25:23.431 INFO:teuthology.orchestra.run.trial162.stdout:506 Cannot talk to daemon 2026-02-13T19:25:23.442 INFO:teuthology.orchestra.run.trial162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T19:25:23.453 INFO:teuthology.orchestra.run.trial162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T19:25:23.500 INFO:teuthology.orchestra.run.trial162.stderr:bash: line 1: ntpq: command not found 2026-02-13T19:25:23.502 INFO:teuthology.orchestra.run.trial162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T19:25:23.502 INFO:teuthology.orchestra.run.trial162.stdout:=============================================================================== 2026-02-13T19:25:23.503 INFO:teuthology.run_tasks:Running task install... 2026-02-13T19:25:23.517 DEBUG:teuthology.task.install:project ceph 2026-02-13T19:25:23.517 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'}} 2026-02-13T19:25:23.517 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'} 2026-02-13T19:25:23.517 INFO:teuthology.task.install:Using flavor: default 2026-02-13T19:25:23.527 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-13T19:25:23.527 INFO:teuthology.task.install:extra packages: [] 2026-02-13T19:25:23.527 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6', 'tag': None, 'wait_for_package': False} 2026-02-13T19:25:23.528 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:25:23.701 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/ 2026-02-13T19:25:23.701 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5230.gafefaa40 2026-02-13T19:25:23.843 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T19:25:23.844 DEBUG:teuthology.orchestra.run.trial162:> set -ex 2026-02-13T19:25:23.844 DEBUG:teuthology.orchestra.run.trial162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T19:25:23.871 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-13T19:25:23.871 DEBUG:teuthology.orchestra.run.trial162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/afefaa40503a4bf024e96f97dd7f852b75f00ad6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T19:25:23.936 DEBUG:teuthology.orchestra.run.trial162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T19:25:24.010 DEBUG:teuthology.orchestra.run.trial162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T19:25:24.074 INFO:teuthology.orchestra.run.trial162.stdout:check_obsoletes = 1 2026-02-13T19:25:24.075 DEBUG:teuthology.orchestra.run.trial162:> sudo yum clean all 2026-02-13T19:25:24.223 INFO:teuthology.orchestra.run.trial162.stdout:41 files removed 2026-02-13T19:25:24.237 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-radosgw 2026-02-13T19:25:24.505 INFO:teuthology.orchestra.run.trial162.stdout:ceph packages for x86_64 734 kB/s | 91 kB 00:00 2026-02-13T19:25:24.646 INFO:teuthology.orchestra.run.trial162.stdout:ceph noarch packages 205 kB/s | 24 kB 00:00 2026-02-13T19:25:24.770 INFO:teuthology.orchestra.run.trial162.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-13T19:25:25.597 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-13T19:25:45.219 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 27 MB 00:19 2026-02-13T19:25:48.586 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 7.8 MB 00:01 2026-02-13T19:25:49.489 INFO:teuthology.orchestra.run.trial162.stdout:CentOS Stream 9 - Extras packages 109 kB/s | 20 kB 00:00 2026-02-13T19:25:49.818 INFO:teuthology.orchestra.run.trial162.stdout:Extra Packages for Enterprise Linux 80 MB/s | 20 MB 00:00 2026-02-13T19:25:53.017 INFO:teuthology.orchestra.run.trial162.stdout:lab-extras 1.6 MB/s | 50 kB 00:00 2026-02-13T19:25:54.296 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:25:54.297 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:25:54.297 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: ceph-radosgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 13 M 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout:Upgrading: 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: librados2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 3.5 M 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: librbd1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.8 M 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: ceph-base x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.2 M 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 24 M 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: ceph-selinux x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T19:25:54.298 INFO:teuthology.orchestra.run.trial162.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: libcephfs2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 909 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: libradosstriper1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 248 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: librgw2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 M 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T19:25:54.299 INFO:teuthology.orchestra.run.trial162.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: python3-ceph-argparse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 45 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: python3-ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 185 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: python3-cephfs x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 203 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: python3-rados x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 324 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: python3-rbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 305 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: python3-rgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 100 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout:Installing weak dependencies: 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:25:54.300 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:25:54.301 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:25:54.301 INFO:teuthology.orchestra.run.trial162.stdout:Install 32 Packages 2026-02-13T19:25:54.301 INFO:teuthology.orchestra.run.trial162.stdout:Upgrade 2 Packages 2026-02-13T19:25:54.301 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:25:54.301 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 67 M 2026-02-13T19:25:54.301 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:25:56.379 INFO:teuthology.orchestra.run.trial162.stdout:(1/34): ceph-base-20.3.0-5230.gafefaa40.el9.x86 2.8 MB/s | 5.2 MB 00:01 2026-02-13T19:25:56.436 INFO:teuthology.orchestra.run.trial162.stdout:(2/34): ceph-selinux-20.3.0-5230.gafefaa40.el9. 447 kB/s | 25 kB 00:00 2026-02-13T19:25:56.629 INFO:teuthology.orchestra.run.trial162.stdout:(3/34): libcephfs2-20.3.0-5230.gafefaa40.el9.x8 4.6 MB/s | 909 kB 00:00 2026-02-13T19:25:56.694 INFO:teuthology.orchestra.run.trial162.stdout:(4/34): libradosstriper1-20.3.0-5230.gafefaa40. 3.8 MB/s | 248 kB 00:00 2026-02-13T19:25:57.487 INFO:teuthology.orchestra.run.trial162.stdout:(5/34): librgw2-20.3.0-5230.gafefaa40.el9.x86_6 8.2 MB/s | 6.5 MB 00:00 2026-02-13T19:25:57.547 INFO:teuthology.orchestra.run.trial162.stdout:(6/34): python3-ceph-argparse-20.3.0-5230.gafef 762 kB/s | 45 kB 00:00 2026-02-13T19:25:57.609 INFO:teuthology.orchestra.run.trial162.stdout:(7/34): python3-ceph-common-20.3.0-5230.gafefaa 2.9 MB/s | 185 kB 00:00 2026-02-13T19:25:57.674 INFO:teuthology.orchestra.run.trial162.stdout:(8/34): python3-cephfs-20.3.0-5230.gafefaa40.el 3.1 MB/s | 203 kB 00:00 2026-02-13T19:25:57.741 INFO:teuthology.orchestra.run.trial162.stdout:(9/34): python3-rados-20.3.0-5230.gafefaa40.el9 4.7 MB/s | 324 kB 00:00 2026-02-13T19:25:57.810 INFO:teuthology.orchestra.run.trial162.stdout:(10/34): python3-rbd-20.3.0-5230.gafefaa40.el9. 4.3 MB/s | 305 kB 00:00 2026-02-13T19:25:57.852 INFO:teuthology.orchestra.run.trial162.stdout:(11/34): ceph-radosgw-20.3.0-5230.gafefaa40.el9 4.0 MB/s | 13 MB 00:03 2026-02-13T19:25:57.871 INFO:teuthology.orchestra.run.trial162.stdout:(12/34): python3-rgw-20.3.0-5230.gafefaa40.el9. 1.6 MB/s | 100 kB 00:00 2026-02-13T19:25:58.427 INFO:teuthology.orchestra.run.trial162.stdout:(13/34): mailcap-2.1.49-5.el9.noarch.rpm 58 kB/s | 33 kB 00:00 2026-02-13T19:25:58.535 INFO:teuthology.orchestra.run.trial162.stdout:(14/34): ceph-common-20.3.0-5230.gafefaa40.el9. 6.0 MB/s | 24 MB 00:04 2026-02-13T19:25:58.536 INFO:teuthology.orchestra.run.trial162.stdout:(15/34): pciutils-3.7.0-7.el9.x86_64.rpm 140 kB/s | 93 kB 00:00 2026-02-13T19:25:59.130 INFO:teuthology.orchestra.run.trial162.stdout:(16/34): boost-program-options-1.75.0-13.el9.x8 148 kB/s | 104 kB 00:00 2026-02-13T19:25:59.258 INFO:teuthology.orchestra.run.trial162.stdout:(17/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 222 kB/s | 160 kB 00:00 2026-02-13T19:25:59.259 INFO:teuthology.orchestra.run.trial162.stdout:(18/34): libnbd-1.20.3-4.el9.x86_64.rpm 226 kB/s | 164 kB 00:00 2026-02-13T19:25:59.715 INFO:teuthology.orchestra.run.trial162.stdout:(19/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 77 kB/s | 45 kB 00:00 2026-02-13T19:26:00.048 INFO:teuthology.orchestra.run.trial162.stdout:(20/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 371 kB/s | 292 kB 00:00 2026-02-13T19:26:00.134 INFO:teuthology.orchestra.run.trial162.stdout:(21/34): librdkafka-1.6.1-102.el9.x86_64.rpm 756 kB/s | 662 kB 00:00 2026-02-13T19:26:00.474 INFO:teuthology.orchestra.run.trial162.stdout:(22/34): qatlib-25.08.0-2.el9.x86_64.rpm 316 kB/s | 240 kB 00:00 2026-02-13T19:26:00.617 INFO:teuthology.orchestra.run.trial162.stdout:(23/34): qatlib-service-25.08.0-2.el9.x86_64.rp 65 kB/s | 37 kB 00:00 2026-02-13T19:26:00.755 INFO:teuthology.orchestra.run.trial162.stdout:(24/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 107 kB/s | 66 kB 00:00 2026-02-13T19:26:00.990 INFO:teuthology.orchestra.run.trial162.stdout:(25/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 596 kB/s | 308 kB 00:00 2026-02-13T19:26:01.031 INFO:teuthology.orchestra.run.trial162.stdout:(26/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 90 kB/s | 25 kB 00:00 2026-02-13T19:26:01.063 INFO:teuthology.orchestra.run.trial162.stdout:(27/34): liboath-2.6.12-1.el9.x86_64.rpm 665 kB/s | 49 kB 00:00 2026-02-13T19:26:01.183 INFO:teuthology.orchestra.run.trial162.stdout:(28/34): libunwind-1.6.2-1.el9.x86_64.rpm 444 kB/s | 67 kB 00:00 2026-02-13T19:26:01.263 INFO:teuthology.orchestra.run.trial162.stdout:(29/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2026-02-13T19:26:01.351 INFO:teuthology.orchestra.run.trial162.stdout:(30/34): re2-20211101-20.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2026-02-13T19:26:01.362 INFO:teuthology.orchestra.run.trial162.stdout:(31/34): libarrow-9.0.0-15.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2026-02-13T19:26:01.429 INFO:teuthology.orchestra.run.trial162.stdout:(32/34): thrift-0.15.0-4.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2026-02-13T19:26:01.861 INFO:teuthology.orchestra.run.trial162.stdout:(33/34): librbd1-20.3.0-5230.gafefaa40.el9.x86_ 5.7 MB/s | 2.8 MB 00:00 2026-02-13T19:26:02.004 INFO:teuthology.orchestra.run.trial162.stdout:(34/34): librados2-20.3.0-5230.gafefaa40.el9.x8 5.3 MB/s | 3.5 MB 00:00 2026-02-13T19:26:02.008 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:02.008 INFO:teuthology.orchestra.run.trial162.stdout:Total 8.7 MB/s | 67 MB 00:07 2026-02-13T19:26:02.092 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:02.110 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:02.110 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:02.371 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:02.371 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:02.673 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:02.829 INFO:teuthology.orchestra.run.trial162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-13T19:26:02.833 INFO:teuthology.orchestra.run.trial162.stdout: Upgrading : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-13T19:26:02.863 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-13T19:26:02.904 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-13T19:26:02.905 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-13T19:26:02.917 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-13T19:26:02.927 INFO:teuthology.orchestra.run.trial162.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-13T19:26:02.929 INFO:teuthology.orchestra.run.trial162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-13T19:26:02.933 INFO:teuthology.orchestra.run.trial162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-13T19:26:02.937 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-13T19:26:02.949 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 9/36 2026-02-13T19:26:02.960 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-13T19:26:02.961 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-13T19:26:02.984 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-13T19:26:02.989 INFO:teuthology.orchestra.run.trial162.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-13T19:26:03.021 INFO:teuthology.orchestra.run.trial162.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-13T19:26:03.025 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-13T19:26:03.032 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-13T19:26:03.035 INFO:teuthology.orchestra.run.trial162.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-13T19:26:03.039 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-13T19:26:03.162 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-13T19:26:03.163 INFO:teuthology.orchestra.run.trial162.stdout: Upgrading : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-13T19:26:03.189 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-13T19:26:03.191 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 20/36 2026-02-13T19:26:03.197 INFO:teuthology.orchestra.run.trial162.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-13T19:26:03.485 INFO:teuthology.orchestra.run.trial162.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-13T19:26:03.486 INFO:teuthology.orchestra.run.trial162.stdout: Installing : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-13T19:26:03.502 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-13T19:26:03.505 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 24/36 2026-02-13T19:26:03.511 INFO:teuthology.orchestra.run.trial162.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-13T19:26:03.513 INFO:teuthology.orchestra.run.trial162.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-13T19:26:03.514 INFO:teuthology.orchestra.run.trial162.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T19:26:03.532 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T19:26:03.535 INFO:teuthology.orchestra.run.trial162.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-13T19:26:03.570 INFO:teuthology.orchestra.run.trial162.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-13T19:26:03.572 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 30/36 2026-02-13T19:26:04.678 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-13T19:26:04.681 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-13T19:26:04.947 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-13T19:26:04.950 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-13T19:26:04.989 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-13T19:26:04.990 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T19:26:04.990 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T19:26:04.990 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:04.993 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout:skipping the directory /sys 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout:skipping the directory /proc 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout:skipping the directory /mnt 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout:skipping the directory /var/tmp 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout:skipping the directory /home 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout:skipping the directory /root 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout:skipping the directory /tmp 2026-02-13T19:26:09.539 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:10.131 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-13T19:26:10.153 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-13T19:26:10.154 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:26:10.154 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T19:26:10.154 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T19:26:10.154 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T19:26:10.154 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:10.156 INFO:teuthology.orchestra.run.trial162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T19:26:10.168 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T19:26:10.168 INFO:teuthology.orchestra.run.trial162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T19:26:10.222 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T19:26:10.222 INFO:teuthology.orchestra.run.trial162.stdout:Creating group 'qat' with GID 992. 2026-02-13T19:26:10.223 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 1/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 5/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 6/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 7/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 8/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 9/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 11/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 12/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 13/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-13T19:26:10.529 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-13T19:26:10.530 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-13T19:26:10.531 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-13T19:26:10.531 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 35/36 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout:Upgraded: 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.593 INFO:teuthology.orchestra.run.trial162.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:10.594 INFO:teuthology.orchestra.run.trial162.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T19:26:10.595 INFO:teuthology.orchestra.run.trial162.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T19:26:10.595 INFO:teuthology.orchestra.run.trial162.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T19:26:10.595 INFO:teuthology.orchestra.run.trial162.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T19:26:10.595 INFO:teuthology.orchestra.run.trial162.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T19:26:10.595 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:10.595 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:10.633 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-test 2026-02-13T19:26:10.932 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:17 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:11.016 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:11.017 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:11.017 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout: ceph-test x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 78 M 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout: libcephsqlite x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 159 k 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T19:26:11.018 INFO:teuthology.orchestra.run.trial162.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout:Install 10 Packages 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:11.019 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 82 M 2026-02-13T19:26:11.020 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 339 M 2026-02-13T19:26:11.020 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:11.638 INFO:teuthology.orchestra.run.trial162.stdout:(1/10): libcephsqlite-20.3.0-5230.gafefaa40.el9 440 kB/s | 159 kB 00:00 2026-02-13T19:26:13.851 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:26:13.851 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:26:15.949 INFO:teuthology.orchestra.run.trial162.stdout:(2/10): ceph-test-20.3.0-5230.gafefaa40.el9.x86 17 MB/s | 78 MB 00:04 2026-02-13T19:26:17.150 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:26:17.150 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:26:17.150 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:26:17.567 INFO:teuthology.orchestra.run.trial162.stdout:(3/10): libatomic-11.5.0-14.el9.x86_64.rpm 3.8 kB/s | 23 kB 00:05 2026-02-13T19:26:17.637 INFO:teuthology.orchestra.run.trial162.stdout:(4/10): libxslt-1.1.34-12.el9.x86_64.rpm 3.2 MB/s | 233 kB 00:00 2026-02-13T19:26:17.680 INFO:teuthology.orchestra.run.trial162.stdout:(5/10): protobuf-3.14.0-17.el9.x86_64.rpm 23 MB/s | 1.0 MB 00:00 2026-02-13T19:26:17.712 INFO:teuthology.orchestra.run.trial162.stdout:(6/10): socat-1.7.4.1-8.el9.x86_64.rpm 9.3 MB/s | 303 kB 00:00 2026-02-13T19:26:17.747 INFO:teuthology.orchestra.run.trial162.stdout:(7/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 64 kB 00:00 2026-02-13T19:26:18.123 INFO:teuthology.orchestra.run.trial162.stdout:(8/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 323 kB/s | 122 kB 00:00 2026-02-13T19:26:19.623 INFO:teuthology.orchestra.run.trial162.stdout:(9/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 257 kB/s | 2.1 MB 00:08 2026-02-13T19:26:19.994 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:26:20.158 INFO:teuthology.orchestra.run.trial162.stdout:(10/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 22 kB/s | 94 kB 00:04 2026-02-13T19:26:20.160 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:20.160 INFO:teuthology.orchestra.run.trial162.stdout:Total 9.0 MB/s | 82 MB 00:09 2026-02-13T19:26:20.225 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:20.236 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:20.236 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:20.520 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:20.520 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:20.828 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:20.836 INFO:teuthology.orchestra.run.trial162.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-13T19:26:20.845 INFO:teuthology.orchestra.run.trial162.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-13T19:26:20.850 INFO:teuthology.orchestra.run.trial162.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-13T19:26:20.854 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-13T19:26:20.859 INFO:teuthology.orchestra.run.trial162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-13T19:26:20.861 INFO:teuthology.orchestra.run.trial162.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-13T19:26:20.870 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-13T19:26:20.879 INFO:teuthology.orchestra.run.trial162.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-13T19:26:20.879 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-13T19:26:24.325 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-13T19:26:24.343 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 1/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-13T19:26:24.661 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:24.716 INFO:teuthology.orchestra.run.trial162.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T19:26:24.717 INFO:teuthology.orchestra.run.trial162.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-13T19:26:24.717 INFO:teuthology.orchestra.run.trial162.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T19:26:24.717 INFO:teuthology.orchestra.run.trial162.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T19:26:24.717 INFO:teuthology.orchestra.run.trial162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T19:26:24.717 INFO:teuthology.orchestra.run.trial162.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-13T19:26:24.717 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:24.717 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:24.754 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph 2026-02-13T19:26:25.051 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:32 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:25.134 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:25.135 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:25.135 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:26:25.135 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 k 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mds x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.4 M 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 936 k 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 297 k 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mon x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.0 M 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph-osd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 166 k 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph-osd-classic x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 16 M 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ceph-osd-crimson x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 19 M 2026-02-13T19:26:25.136 INFO:teuthology.orchestra.run.trial162.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T19:26:25.137 INFO:teuthology.orchestra.run.trial162.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-collections 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T19:26:25.138 INFO:teuthology.orchestra.run.trial162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:25.139 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:25.140 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:25.140 INFO:teuthology.orchestra.run.trial162.stdout:Install 42 Packages 2026-02-13T19:26:25.140 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:25.140 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 48 M 2026-02-13T19:26:25.141 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 167 M 2026-02-13T19:26:25.141 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:26.070 INFO:teuthology.orchestra.run.trial162.stdout:(1/42): ceph-20.3.0-5230.gafefaa40.el9.x86_64.r 107 kB/s | 6.5 kB 00:00 2026-02-13T19:26:26.156 INFO:teuthology.orchestra.run.trial162.stdout:(2/42): ceph-mgr-20.3.0-5230.gafefaa40.el9.x86_ 6.2 MB/s | 936 kB 00:00 2026-02-13T19:26:26.183 INFO:teuthology.orchestra.run.trial162.stdout:(3/42): ceph-mds-20.3.0-5230.gafefaa40.el9.x86_ 14 MB/s | 2.4 MB 00:00 2026-02-13T19:26:26.186 INFO:teuthology.orchestra.run.trial162.stdout:(4/42): ceph-osd-20.3.0-5230.gafefaa40.el9.x86_ 5.3 MB/s | 166 kB 00:00 2026-02-13T19:26:26.351 INFO:teuthology.orchestra.run.trial162.stdout:(5/42): ceph-mon-20.3.0-5230.gafefaa40.el9.x86_ 18 MB/s | 5.0 MB 00:00 2026-02-13T19:26:26.400 INFO:teuthology.orchestra.run.trial162.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5230.gafef 5.9 MB/s | 297 kB 00:00 2026-02-13T19:26:26.470 INFO:teuthology.orchestra.run.trial162.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 580 kB/s | 40 kB 00:00 2026-02-13T19:26:26.493 INFO:teuthology.orchestra.run.trial162.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 3.1 MB/s | 72 kB 00:00 2026-02-13T19:26:26.538 INFO:teuthology.orchestra.run.trial162.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.5 MB/s | 253 kB 00:00 2026-02-13T19:26:26.607 INFO:teuthology.orchestra.run.trial162.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 18 MB/s | 1.2 MB 00:00 2026-02-13T19:26:26.631 INFO:teuthology.orchestra.run.trial162.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 4.4 MB/s | 106 kB 00:00 2026-02-13T19:26:26.656 INFO:teuthology.orchestra.run.trial162.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 5.5 MB/s | 135 kB 00:00 2026-02-13T19:26:26.680 INFO:teuthology.orchestra.run.trial162.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 6.1 MB/s | 150 kB 00:00 2026-02-13T19:26:26.704 INFO:teuthology.orchestra.run.trial162.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 5.0 MB/s | 126 kB 00:00 2026-02-13T19:26:26.738 INFO:teuthology.orchestra.run.trial162.stdout:(15/42): ceph-osd-classic-20.3.0-5230.gafefaa40 28 MB/s | 16 MB 00:00 2026-02-13T19:26:26.739 INFO:teuthology.orchestra.run.trial162.stdout:(16/42): python3-urllib3-1.26.5-7.el9.noarch.rp 6.1 MB/s | 218 kB 00:00 2026-02-13T19:26:26.766 INFO:teuthology.orchestra.run.trial162.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 9.8 MB/s | 266 kB 00:00 2026-02-13T19:26:26.822 INFO:teuthology.orchestra.run.trial162.stdout:(18/42): unzip-6.0-59.el9.x86_64.rpm 2.1 MB/s | 182 kB 00:00 2026-02-13T19:26:26.871 INFO:teuthology.orchestra.run.trial162.stdout:(19/42): ceph-osd-crimson-20.3.0-5230.gafefaa40 28 MB/s | 19 MB 00:00 2026-02-13T19:26:27.559 INFO:teuthology.orchestra.run.trial162.stdout:(20/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 311 kB/s | 246 kB 00:00 2026-02-13T19:26:27.580 INFO:teuthology.orchestra.run.trial162.stdout:(21/42): lua-5.4.4-4.el9.x86_64.rpm 249 kB/s | 188 kB 00:00 2026-02-13T19:26:27.601 INFO:teuthology.orchestra.run.trial162.stdout:(22/42): python3-libstoragemgmt-1.10.1-1.el9.x8 242 kB/s | 177 kB 00:00 2026-02-13T19:26:27.678 INFO:teuthology.orchestra.run.trial162.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 291 kB/s | 22 kB 00:00 2026-02-13T19:26:28.012 INFO:teuthology.orchestra.run.trial162.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 454 kB/s | 151 kB 00:00 2026-02-13T19:26:28.054 INFO:teuthology.orchestra.run.trial162.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 711 kB/s | 29 kB 00:00 2026-02-13T19:26:28.095 INFO:teuthology.orchestra.run.trial162.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 1.4 MB/s | 60 kB 00:00 2026-02-13T19:26:28.137 INFO:teuthology.orchestra.run.trial162.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2026-02-13T19:26:28.152 INFO:teuthology.orchestra.run.trial162.stdout:(28/42): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2026-02-13T19:26:28.192 INFO:teuthology.orchestra.run.trial162.stdout:(29/42): python3-packaging-20.9-5.el9.noarch.rp 122 kB/s | 77 kB 00:00 2026-02-13T19:26:28.219 INFO:teuthology.orchestra.run.trial162.stdout:(30/42): python3-cheroot-10.0.1-4.el9.noarch.rp 2.1 MB/s | 173 kB 00:00 2026-02-13T19:26:28.261 INFO:teuthology.orchestra.run.trial162.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 430 kB/s | 18 kB 00:00 2026-02-13T19:26:28.275 INFO:teuthology.orchestra.run.trial162.stdout:(32/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 129 kB/s | 11 kB 00:00 2026-02-13T19:26:28.301 INFO:teuthology.orchestra.run.trial162.stdout:(33/42): python3-jaraco-collections-3.0.0-8.el9 563 kB/s | 23 kB 00:00 2026-02-13T19:26:28.317 INFO:teuthology.orchestra.run.trial162.stdout:(34/42): python3-jaraco-context-6.0.1-3.el9.noa 467 kB/s | 20 kB 00:00 2026-02-13T19:26:28.343 INFO:teuthology.orchestra.run.trial162.stdout:(35/42): python3-jaraco-functools-3.5.0-2.el9.n 474 kB/s | 19 kB 00:00 2026-02-13T19:26:28.359 INFO:teuthology.orchestra.run.trial162.stdout:(36/42): python3-jaraco-text-4.0.0-2.el9.noarch 624 kB/s | 26 kB 00:00 2026-02-13T19:26:28.385 INFO:teuthology.orchestra.run.trial162.stdout:(37/42): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2026-02-13T19:26:28.389 INFO:teuthology.orchestra.run.trial162.stdout:(38/42): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2026-02-13T19:26:28.401 INFO:teuthology.orchestra.run.trial162.stdout:(39/42): python3-portend-3.1.0-2.el9.noarch.rpm 390 kB/s | 16 kB 00:00 2026-02-13T19:26:28.427 INFO:teuthology.orchestra.run.trial162.stdout:(40/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2026-02-13T19:26:28.431 INFO:teuthology.orchestra.run.trial162.stdout:(41/42): python3-tempora-5.0.0-2.el9.noarch.rpm 855 kB/s | 36 kB 00:00 2026-02-13T19:26:28.444 INFO:teuthology.orchestra.run.trial162.stdout:(42/42): python3-zc-lockfile-2.0-10.el9.noarch. 468 kB/s | 20 kB 00:00 2026-02-13T19:26:28.445 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:28.445 INFO:teuthology.orchestra.run.trial162.stdout:Total 15 MB/s | 48 MB 00:03 2026-02-13T19:26:28.590 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:28.609 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:28.609 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:28.804 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:28.804 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:29.031 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:29.033 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-13T19:26:29.038 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-13T19:26:29.042 INFO:teuthology.orchestra.run.trial162.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-13T19:26:29.051 INFO:teuthology.orchestra.run.trial162.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-13T19:26:29.051 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-13T19:26:29.074 INFO:teuthology.orchestra.run.trial162.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-13T19:26:29.077 INFO:teuthology.orchestra.run.trial162.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-13T19:26:29.082 INFO:teuthology.orchestra.run.trial162.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-13T19:26:29.087 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-13T19:26:29.093 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-13T19:26:29.097 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-13T19:26:29.103 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-13T19:26:29.108 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-13T19:26:29.111 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-13T19:26:29.117 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-13T19:26:29.120 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-13T19:26:29.127 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-13T19:26:29.131 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-13T19:26:29.153 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-13T19:26:29.166 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-13T19:26:29.171 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-13T19:26:29.181 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-13T19:26:29.187 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-13T19:26:29.201 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-13T19:26:29.216 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-13T19:26:29.292 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-13T19:26:29.301 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-13T19:26:29.324 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-13T19:26:29.353 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-13T19:26:29.359 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-13T19:26:29.410 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-13T19:26:29.453 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 32/42 2026-02-13T19:26:29.454 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-13T19:26:29.477 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-13T19:26:29.478 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:26:29.478 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T19:26:29.478 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T19:26:29.478 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T19:26:29.478 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:29.482 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-13T19:26:29.497 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-13T19:26:29.498 INFO:teuthology.orchestra.run.trial162.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T19:26:29.498 INFO:teuthology.orchestra.run.trial162.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T19:26:29.498 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:29.505 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-13T19:26:29.528 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-13T19:26:29.529 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T19:26:29.529 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:30.177 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-13T19:26:30.950 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 37/42 2026-02-13T19:26:30.960 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 38/42 2026-02-13T19:26:30.961 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-13T19:26:30.987 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-13T19:26:30.987 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:26:30.987 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T19:26:30.987 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T19:26:30.987 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T19:26:30.987 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:31.203 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-13T19:26:31.224 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-13T19:26:31.224 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:26:31.224 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T19:26:31.224 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T19:26:31.224 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T19:26:31.224 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:31.321 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-13T19:26:31.343 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-13T19:26:31.343 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:26:31.343 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T19:26:31.343 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T19:26:31.343 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T19:26:31.343 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:31.344 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-13T19:26:31.351 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-13T19:26:31.376 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-13T19:26:31.764 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 1/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 3/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 4/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 5/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 6/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 7/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 8/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-13T19:26:31.765 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-13T19:26:31.766 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-13T19:26:31.767 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-13T19:26:31.768 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-13T19:26:31.768 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-13T19:26:31.768 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-13T19:26:31.768 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-13T19:26:31.768 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T19:26:31.821 INFO:teuthology.orchestra.run.trial162.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T19:26:31.822 INFO:teuthology.orchestra.run.trial162.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: zip-3.0-35.el9.x86_64 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:31.823 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:31.859 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-base 2026-02-13T19:26:32.161 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:39 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:32.226 INFO:teuthology.orchestra.run.trial162.stdout:Package ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:26:32.244 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:32.244 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:26:32.244 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:32.267 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install cephadm 2026-02-13T19:26:32.563 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:39 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:32.647 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:32.647 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:32.647 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout: cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 1.0 M 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:Install 1 Package 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 1.0 M 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 1.0 M 2026-02-13T19:26:32.648 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:32.768 INFO:teuthology.orchestra.run.trial162.stdout:cephadm-20.3.0-5230.gafefaa40.el9.noarch.rpm 8.5 MB/s | 1.0 MB 00:00 2026-02-13T19:26:32.768 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:32.768 INFO:teuthology.orchestra.run.trial162.stdout:Total 8.4 MB/s | 1.0 MB 00:00 2026-02-13T19:26:32.768 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:32.770 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:32.771 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:32.774 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:32.775 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:32.793 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:32.855 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:26:32.869 INFO:teuthology.orchestra.run.trial162.stdout: Installing : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:26:32.933 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:26:32.986 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:26:32.986 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:32.986 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:32.986 INFO:teuthology.orchestra.run.trial162.stdout: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:32.986 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:32.986 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:33.013 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-immutable-object-cache 2026-02-13T19:26:33.326 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:40 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:33.409 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:33.409 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repo Size 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 147 k 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout:Install 1 Package 2026-02-13T19:26:33.410 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:33.411 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 147 k 2026-02-13T19:26:33.411 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 419 k 2026-02-13T19:26:33.411 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:33.495 INFO:teuthology.orchestra.run.trial162.stdout:ceph-immutable-object-cache-20.3.0-5230.gafefaa 1.7 MB/s | 147 kB 00:00 2026-02-13T19:26:33.495 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:33.495 INFO:teuthology.orchestra.run.trial162.stdout:Total 1.7 MB/s | 147 kB 00:00 2026-02-13T19:26:33.495 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:33.499 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:33.499 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:33.525 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:33.525 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:33.574 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:33.576 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-13T19:26:33.598 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-13T19:26:33.598 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:26:33.598 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T19:26:33.598 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:33.904 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-13T19:26:33.905 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:33.905 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:33.905 INFO:teuthology.orchestra.run.trial162.stdout: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:26:33.905 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:33.905 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:33.932 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-mgr 2026-02-13T19:26:34.230 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:41 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:34.294 INFO:teuthology.orchestra.run.trial162.stdout:Package ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:26:34.312 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:34.312 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:26:34.313 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:34.333 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-mgr-dashboard 2026-02-13T19:26:34.628 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:41 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 51 M 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 57 k 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 17 k 2026-02-13T19:26:34.710 INFO:teuthology.orchestra.run.trial162.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout:Install 13 Packages 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 55 M 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 262 M 2026-02-13T19:26:34.711 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:35.046 INFO:teuthology.orchestra.run.trial162.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5230.gafe 257 kB/s | 17 kB 00:00 2026-02-13T19:26:35.051 INFO:teuthology.orchestra.run.trial162.stdout:(2/13): ceph-grafana-dashboards-20.3.0-5230.gaf 791 kB/s | 57 kB 00:00 2026-02-13T19:26:35.549 INFO:teuthology.orchestra.run.trial162.stdout:(3/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 95 kB/s | 48 kB 00:00 2026-02-13T19:26:35.617 INFO:teuthology.orchestra.run.trial162.stdout:(4/13): protobuf-compiler-3.14.0-17.el9.x86_64. 12 MB/s | 862 kB 00:00 2026-02-13T19:26:35.778 INFO:teuthology.orchestra.run.trial162.stdout:(5/13): python3-protobuf-3.14.0-17.el9.noarch.r 368 kB/s | 267 kB 00:00 2026-02-13T19:26:35.828 INFO:teuthology.orchestra.run.trial162.stdout:(6/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.6 MB/s | 551 kB 00:00 2026-02-13T19:26:35.897 INFO:teuthology.orchestra.run.trial162.stdout:(7/13): grpc-data-1.46.7-10.el9.noarch.rpm 165 kB/s | 19 kB 00:00 2026-02-13T19:26:35.907 INFO:teuthology.orchestra.run.trial162.stdout:(8/13): python3-grpcio-1.46.7-10.el9.x86_64.rpm 26 MB/s | 2.0 MB 00:00 2026-02-13T19:26:35.930 INFO:teuthology.orchestra.run.trial162.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 1.3 MB/s | 31 kB 00:00 2026-02-13T19:26:36.009 INFO:teuthology.orchestra.run.trial162.stdout:(10/13): ceph-mgr-dashboard-20.3.0-5230.gafefaa 49 MB/s | 51 MB 00:01 2026-02-13T19:26:36.010 INFO:teuthology.orchestra.run.trial162.stdout:(11/13): python3-routes-2.5.1-5.el9.noarch.rpm 2.3 MB/s | 188 kB 00:00 2026-02-13T19:26:36.039 INFO:teuthology.orchestra.run.trial162.stdout:(12/13): python3-grpcio-tools-1.46.7-10.el9.x86 1.0 MB/s | 144 kB 00:00 2026-02-13T19:26:36.124 INFO:teuthology.orchestra.run.trial162.stdout:(13/13): python3-xmltodict-0.12.0-15.el9.noarch 192 kB/s | 22 kB 00:00 2026-02-13T19:26:36.131 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:36.131 INFO:teuthology.orchestra.run.trial162.stdout:Total 39 MB/s | 55 MB 00:01 2026-02-13T19:26:36.159 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:36.169 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:36.169 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:36.378 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:36.379 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:36.604 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:36.609 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-13T19:26:36.620 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-13T19:26:36.622 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-13T19:26:36.650 INFO:teuthology.orchestra.run.trial162.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-13T19:26:36.694 INFO:teuthology.orchestra.run.trial162.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-13T19:26:36.704 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-13T19:26:36.732 INFO:teuthology.orchestra.run.trial162.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-13T19:26:36.742 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-13T19:26:36.749 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-13T19:26:36.752 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-13T19:26:36.761 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 11/13 2026-02-13T19:26:39.770 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 12/13 2026-02-13T19:26:39.777 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-13T19:26:40.041 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 1/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 2/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 3/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-13T19:26:40.042 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:40.096 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:40.132 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-13T19:26:40.428 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:47 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:40.512 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:40.512 INFO:teuthology.orchestra.run.trial162.stdout:======================================================================================= 2026-02-13T19:26:40.512 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:26:40.512 INFO:teuthology.orchestra.run.trial162.stdout:======================================================================================= 2026-02-13T19:26:40.512 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 7.4 M 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout:======================================================================================= 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout:Install 12 Packages 2026-02-13T19:26:40.513 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:40.514 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 43 M 2026-02-13T19:26:40.514 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 234 M 2026-02-13T19:26:40.514 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:41.138 INFO:teuthology.orchestra.run.trial162.stdout:(1/12): ceph-mgr-diskprediction-local-20.3.0-52 33 MB/s | 7.4 MB 00:00 2026-02-13T19:26:41.211 INFO:teuthology.orchestra.run.trial162.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 450 kB/s | 33 kB 00:00 2026-02-13T19:26:41.265 INFO:teuthology.orchestra.run.trial162.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 55 MB/s | 3.0 MB 00:00 2026-02-13T19:26:41.291 INFO:teuthology.orchestra.run.trial162.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 710 kB/s | 18 kB 00:00 2026-02-13T19:26:41.318 INFO:teuthology.orchestra.run.trial162.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 1.5 MB/s | 42 kB 00:00 2026-02-13T19:26:41.379 INFO:teuthology.orchestra.run.trial162.stdout:(6/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 87 MB/s | 5.3 MB 00:00 2026-02-13T19:26:41.408 INFO:teuthology.orchestra.run.trial162.stdout:(7/12): python3-devel-3.9.25-2.el9.x86_64.rpm 8.2 MB/s | 244 kB 00:00 2026-02-13T19:26:41.484 INFO:teuthology.orchestra.run.trial162.stdout:(8/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 80 MB/s | 6.1 MB 00:00 2026-02-13T19:26:41.517 INFO:teuthology.orchestra.run.trial162.stdout:(9/12): python3-numpy-f2py-1.23.5-2.el9.x86_64. 13 MB/s | 442 kB 00:00 2026-02-13T19:26:41.652 INFO:teuthology.orchestra.run.trial162.stdout:(10/12): libquadmath-11.5.0-14.el9.x86_64.rpm 250 kB/s | 184 kB 00:00 2026-02-13T19:26:41.702 INFO:teuthology.orchestra.run.trial162.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 105 MB/s | 19 MB 00:00 2026-02-13T19:26:41.832 INFO:teuthology.orchestra.run.trial162.stdout:(12/12): libgfortran-11.5.0-14.el9.x86_64.rpm 864 kB/s | 794 kB 00:00 2026-02-13T19:26:41.833 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:41.833 INFO:teuthology.orchestra.run.trial162.stdout:Total 32 MB/s | 43 MB 00:01 2026-02-13T19:26:41.990 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:42.004 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:42.004 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:42.207 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:42.208 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:42.427 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:42.430 INFO:teuthology.orchestra.run.trial162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2026-02-13T19:26:42.436 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-13T19:26:42.470 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-13T19:26:42.474 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 4/12 2026-02-13T19:26:42.534 INFO:teuthology.orchestra.run.trial162.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-13T19:26:42.537 INFO:teuthology.orchestra.run.trial162.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-13T19:26:42.558 INFO:teuthology.orchestra.run.trial162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2026-02-13T19:26:42.885 INFO:teuthology.orchestra.run.trial162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2026-02-13T19:26:42.958 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-13T19:26:43.633 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T19:26:44.090 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-13T19:26:44.091 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-13T19:26:44.386 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-13T19:26:44.387 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 1/12 2026-02-13T19:26:44.387 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-13T19:26:44.387 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-13T19:26:44.387 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2026-02-13T19:26:44.387 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2026-02-13T19:26:44.387 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2026-02-13T19:26:44.387 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-13T19:26:44.388 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-13T19:26:44.388 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 9/12 2026-02-13T19:26:44.388 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T19:26:44.388 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-13T19:26:44.443 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-13T19:26:44.443 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:44.444 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:44.444 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:44.444 INFO:teuthology.orchestra.run.trial162.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T19:26:44.444 INFO:teuthology.orchestra.run.trial162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T19:26:44.444 INFO:teuthology.orchestra.run.trial162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T19:26:44.444 INFO:teuthology.orchestra.run.trial162.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T19:26:44.445 INFO:teuthology.orchestra.run.trial162.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T19:26:44.445 INFO:teuthology.orchestra.run.trial162.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T19:26:44.445 INFO:teuthology.orchestra.run.trial162.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T19:26:44.445 INFO:teuthology.orchestra.run.trial162.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T19:26:44.445 INFO:teuthology.orchestra.run.trial162.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T19:26:44.445 INFO:teuthology.orchestra.run.trial162.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T19:26:44.446 INFO:teuthology.orchestra.run.trial162.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T19:26:44.446 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:44.446 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:44.480 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-mgr-rook 2026-02-13T19:26:44.774 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:51 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:44.857 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:44.857 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:44.857 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:26:44.857 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-rook noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 50 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T19:26:44.858 INFO:teuthology.orchestra.run.trial162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T19:26:44.859 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:44.859 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:44.859 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:44.859 INFO:teuthology.orchestra.run.trial162.stdout:Install 10 Packages 2026-02-13T19:26:44.859 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:44.860 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 2.0 M 2026-02-13T19:26:44.860 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 25 M 2026-02-13T19:26:44.860 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:45.102 INFO:teuthology.orchestra.run.trial162.stdout:(1/10): ceph-mgr-rook-20.3.0-5230.gafefaa40.el9 683 kB/s | 50 kB 00:00 2026-02-13T19:26:47.643 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] python3-pyasn1-0.4.8-6.el9.noarch.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-pyasn1-0.4.8-6.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:26:47.643 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] python3-pyasn1-modules-0.4.8-6.el9.noarch.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-pyasn1-modules-0.4.8-6.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:26:47.643 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] python3-requests-oauthlib-1.3.0-12.el9.noarch.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-requests-oauthlib-1.3.0-12.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:26:50.846 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] python3-pyasn1-0.4.8-6.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-pyasn1-0.4.8-6.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:26:50.846 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] python3-pyasn1-modules-0.4.8-6.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-pyasn1-modules-0.4.8-6.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:26:50.846 INFO:teuthology.orchestra.run.trial162.stdout:[MIRROR] python3-requests-oauthlib-1.3.0-12.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-requests-oauthlib-1.3.0-12.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:26:51.421 INFO:teuthology.orchestra.run.trial162.stdout:(2/10): python3-requests-oauthlib-1.3.0-12.el9. 8.5 kB/s | 54 kB 00:06 2026-02-13T19:26:51.508 INFO:teuthology.orchestra.run.trial162.stdout:(3/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 24 kB/s | 159 kB 00:06 2026-02-13T19:26:51.510 INFO:teuthology.orchestra.run.trial162.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 359 kB/s | 32 kB 00:00 2026-02-13T19:26:51.568 INFO:teuthology.orchestra.run.trial162.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 236 kB/s | 14 kB 00:00 2026-02-13T19:26:51.583 INFO:teuthology.orchestra.run.trial162.stdout:(6/10): python3-google-auth-2.45.0-1.el9.noarch 3.4 MB/s | 254 kB 00:00 2026-02-13T19:26:51.611 INFO:teuthology.orchestra.run.trial162.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 2.1 MB/s | 59 kB 00:00 2026-02-13T19:26:51.641 INFO:teuthology.orchestra.run.trial162.stdout:(8/10): python3-websocket-client-1.2.3-2.el9.no 3.0 MB/s | 90 kB 00:00 2026-02-13T19:26:51.665 INFO:teuthology.orchestra.run.trial162.stdout:(9/10): python3-pyasn1-modules-0.4.8-6.el9.noar 42 kB/s | 279 kB 00:06 2026-02-13T19:26:51.689 INFO:teuthology.orchestra.run.trial162.stdout:(10/10): python3-kubernetes-26.1.0-3.el9.noarch 8.5 MB/s | 1.0 MB 00:00 2026-02-13T19:26:51.691 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:51.691 INFO:teuthology.orchestra.run.trial162.stdout:Total 298 kB/s | 2.0 MB 00:06 2026-02-13T19:26:51.716 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:51.723 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:51.723 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:51.762 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:51.762 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:51.836 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:51.886 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-13T19:26:51.899 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-13T19:26:51.913 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-13T19:26:51.918 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-13T19:26:51.926 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-13T19:26:51.965 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-13T19:26:51.977 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T19:26:52.293 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-13T19:26:52.307 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-13T19:26:52.308 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-13T19:26:52.416 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 1/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-13T19:26:52.417 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-13T19:26:52.472 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-13T19:26:52.472 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:52.472 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:52.472 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:52.472 INFO:teuthology.orchestra.run.trial162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T19:26:52.472 INFO:teuthology.orchestra.run.trial162.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:52.473 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:52.503 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-mgr-cephadm 2026-02-13T19:26:52.802 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:52.885 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:52.886 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:52.886 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:26:52.886 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:52.886 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:26:52.886 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 184 k 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:26:52.887 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:26:52.888 INFO:teuthology.orchestra.run.trial162.stdout:Install 7 Packages 2026-02-13T19:26:52.888 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:52.888 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 7.1 M 2026-02-13T19:26:52.888 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 34 M 2026-02-13T19:26:52.888 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:26:53.118 INFO:teuthology.orchestra.run.trial162.stdout:(1/7): ceph-mgr-cephadm-20.3.0-5230.gafefaa40.e 1.8 MB/s | 184 kB 00:00 2026-02-13T19:26:53.538 INFO:teuthology.orchestra.run.trial162.stdout:(2/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 83 kB/s | 35 kB 00:00 2026-02-13T19:26:53.688 INFO:teuthology.orchestra.run.trial162.stdout:(3/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.6 MB/s | 548 kB 00:00 2026-02-13T19:26:53.703 INFO:teuthology.orchestra.run.trial162.stdout:(4/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 364 kB/s | 249 kB 00:00 2026-02-13T19:26:53.710 INFO:teuthology.orchestra.run.trial162.stdout:(5/7): python3-natsort-7.1.1-5.el9.noarch.rpm 2.6 MB/s | 58 kB 00:00 2026-02-13T19:26:53.782 INFO:teuthology.orchestra.run.trial162.stdout:(6/7): python3-typing-extensions-4.15.0-1.el9.n 1.1 MB/s | 86 kB 00:00 2026-02-13T19:26:58.594 INFO:teuthology.orchestra.run.trial162.stdout:(7/7): python3-babel-2.9.1-2.el9.noarch.rpm 1.1 MB/s | 6.0 MB 00:05 2026-02-13T19:26:58.596 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:26:58.596 INFO:teuthology.orchestra.run.trial162.stdout:Total 1.2 MB/s | 7.1 MB 00:05 2026-02-13T19:26:58.634 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:26:58.641 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:26:58.641 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:26:58.712 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:26:58.712 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:26:58.801 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:26:58.850 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-13T19:26:58.860 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-13T19:26:58.866 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-13T19:26:59.089 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-13T19:26:59.112 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-13T19:26:59.138 INFO:teuthology.orchestra.run.trial162.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-13T19:26:59.139 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-13T19:26:59.250 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-13T19:26:59.250 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/7 2026-02-13T19:26:59.250 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-13T19:26:59.250 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-13T19:26:59.250 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-13T19:26:59.250 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-13T19:26:59.251 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T19:26:59.306 INFO:teuthology.orchestra.run.trial162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T19:26:59.307 INFO:teuthology.orchestra.run.trial162.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T19:26:59.307 INFO:teuthology.orchestra.run.trial162.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T19:26:59.307 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:26:59.307 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:59.337 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-osd 2026-02-13T19:26:59.640 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:26:59.705 INFO:teuthology.orchestra.run.trial162.stdout:Package ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:26:59.723 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:26:59.724 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:26:59.724 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:26:59.745 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-osd-classic 2026-02-13T19:27:00.047 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:00.114 INFO:teuthology.orchestra.run.trial162.stdout:Package ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:00.132 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:00.132 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:00.132 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:00.153 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-fuse 2026-02-13T19:27:00.449 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:00.532 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repo Size 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout: ceph-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 987 k 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:27:00.533 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:00.534 INFO:teuthology.orchestra.run.trial162.stdout:Install 1 Package 2026-02-13T19:27:00.534 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:00.534 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 987 k 2026-02-13T19:27:00.534 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 2.9 M 2026-02-13T19:27:00.534 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:27:00.664 INFO:teuthology.orchestra.run.trial162.stdout:ceph-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 7.4 MB/s | 987 kB 00:00 2026-02-13T19:27:00.665 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:27:00.665 INFO:teuthology.orchestra.run.trial162.stdout:Total 7.3 MB/s | 987 kB 00:00 2026-02-13T19:27:00.665 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:27:00.669 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:27:00.669 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:27:00.705 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:27:00.705 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:27:00.797 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:27:00.814 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:01.056 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:01.111 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:01.111 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:01.112 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:27:01.112 INFO:teuthology.orchestra.run.trial162.stdout: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:27:01.112 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:01.112 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:01.139 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install ceph-volume 2026-02-13T19:27:01.439 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:01.524 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:01.524 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:01.524 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repository Size 2026-02-13T19:27:01.524 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout: ceph-volume noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 299 k 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout:Install 2 Packages 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 649 k 2026-02-13T19:27:01.525 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 2.2 M 2026-02-13T19:27:01.526 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:27:01.927 INFO:teuthology.orchestra.run.trial162.stdout:(1/2): ceph-volume-20.3.0-5230.gafefaa40.el9.no 3.1 MB/s | 299 kB 00:00 2026-02-13T19:27:03.358 INFO:teuthology.orchestra.run.trial162.stdout:(2/2): cryptsetup-2.8.1-2.el9.x86_64.rpm 230 kB/s | 350 kB 00:01 2026-02-13T19:27:03.359 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:27:03.359 INFO:teuthology.orchestra.run.trial162.stdout:Total 354 kB/s | 649 kB 00:01 2026-02-13T19:27:03.368 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:27:03.375 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:27:03.375 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:27:03.418 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:27:03.419 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:27:03.498 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:27:03.561 INFO:teuthology.orchestra.run.trial162.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 1/2 2026-02-13T19:27:03.562 INFO:teuthology.orchestra.run.trial162.stdout: Installing : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-13T19:27:03.577 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-13T19:27:03.577 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:27:03.577 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-13T19:27:03.577 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:03.799 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 1/2 2026-02-13T19:27:03.856 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 2/2 2026-02-13T19:27:03.857 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:03.857 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:27:03.857 INFO:teuthology.orchestra.run.trial162.stdout: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch cryptsetup-2.8.1-2.el9.x86_64 2026-02-13T19:27:03.857 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:03.857 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:03.884 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install librados-devel 2026-02-13T19:27:04.191 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:11 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:04.276 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:04.276 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:04.276 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repo Size 2026-02-13T19:27:04.276 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:04.276 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout: librados-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 123 k 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout:Install 1 Package 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 123 k 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 437 k 2026-02-13T19:27:04.277 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:27:04.361 INFO:teuthology.orchestra.run.trial162.stdout:librados-devel-20.3.0-5230.gafefaa40.el9.x86_64 1.4 MB/s | 123 kB 00:00 2026-02-13T19:27:04.362 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:27:04.362 INFO:teuthology.orchestra.run.trial162.stdout:Total 1.4 MB/s | 123 kB 00:00 2026-02-13T19:27:04.362 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:27:04.364 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:27:04.364 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:27:04.396 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:27:04.396 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:27:04.449 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:27:04.465 INFO:teuthology.orchestra.run.trial162.stdout: Installing : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:04.552 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:04.609 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:04.609 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:04.609 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:27:04.609 INFO:teuthology.orchestra.run.trial162.stdout: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:27:04.609 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:04.609 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:04.639 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install libcephfs2 2026-02-13T19:27:04.942 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:11 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:05.007 INFO:teuthology.orchestra.run.trial162.stdout:Package libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:05.025 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:05.025 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:05.025 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:05.047 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install libcephfs-devel 2026-02-13T19:27:05.344 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:12 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repo Size 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout: libcephfs-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 37 k 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout:Installing dependencies: 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:05.427 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:27:05.428 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:05.428 INFO:teuthology.orchestra.run.trial162.stdout:Install 2 Packages 2026-02-13T19:27:05.428 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:05.428 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 62 k 2026-02-13T19:27:05.428 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 223 k 2026-02-13T19:27:05.428 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:27:05.491 INFO:teuthology.orchestra.run.trial162.stdout:(1/2): libcephfs-proxy2-20.3.0-5230.gafefaa40.e 399 kB/s | 25 kB 00:00 2026-02-13T19:27:05.493 INFO:teuthology.orchestra.run.trial162.stdout:(2/2): libcephfs-devel-20.3.0-5230.gafefaa40.el 575 kB/s | 37 kB 00:00 2026-02-13T19:27:05.494 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:27:05.494 INFO:teuthology.orchestra.run.trial162.stdout:Total 937 kB/s | 62 kB 00:00 2026-02-13T19:27:05.494 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:27:05.497 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:27:05.497 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:27:05.530 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:27:05.531 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:27:05.579 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:27:05.579 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-13T19:27:05.595 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-13T19:27:05.612 INFO:teuthology.orchestra.run.trial162.stdout: Installing : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-13T19:27:05.663 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-13T19:27:05.663 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-13T19:27:05.719 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-13T19:27:05.720 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:05.720 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:27:05.720 INFO:teuthology.orchestra.run.trial162.stdout: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:27:05.720 INFO:teuthology.orchestra.run.trial162.stdout: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:27:05.720 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:05.720 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:05.746 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install librados2 2026-02-13T19:27:06.047 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:06.111 INFO:teuthology.orchestra.run.trial162.stdout:Package librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:06.129 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:06.129 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:06.129 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:06.151 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install librbd1 2026-02-13T19:27:06.453 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:06.517 INFO:teuthology.orchestra.run.trial162.stdout:Package librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:06.535 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:06.536 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:06.536 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:06.557 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install python3-rados 2026-02-13T19:27:06.857 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:06.922 INFO:teuthology.orchestra.run.trial162.stdout:Package python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:06.940 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:06.941 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:06.941 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:06.962 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install python3-rgw 2026-02-13T19:27:07.263 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:14 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:07.327 INFO:teuthology.orchestra.run.trial162.stdout:Package python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:07.345 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:07.346 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:07.346 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:07.367 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install python3-cephfs 2026-02-13T19:27:07.667 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:14 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:07.732 INFO:teuthology.orchestra.run.trial162.stdout:Package python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:07.751 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:07.751 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:07.751 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:07.772 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install python3-rbd 2026-02-13T19:27:08.071 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:15 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:08.137 INFO:teuthology.orchestra.run.trial162.stdout:Package python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:27:08.156 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:08.157 INFO:teuthology.orchestra.run.trial162.stdout:Nothing to do. 2026-02-13T19:27:08.157 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:08.178 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install rbd-fuse 2026-02-13T19:27:08.473 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:15 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout: Package Architecture Version Repository Size 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout: rbd-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 84 k 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:08.556 INFO:teuthology.orchestra.run.trial162.stdout:Install 1 Package 2026-02-13T19:27:08.557 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:08.557 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 84 k 2026-02-13T19:27:08.557 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 209 k 2026-02-13T19:27:08.557 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:27:08.632 INFO:teuthology.orchestra.run.trial162.stdout:rbd-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2026-02-13T19:27:08.632 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:27:08.632 INFO:teuthology.orchestra.run.trial162.stdout:Total 1.1 MB/s | 84 kB 00:00 2026-02-13T19:27:08.632 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:27:08.636 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:27:08.636 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:27:08.668 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:27:08.668 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:27:08.718 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:27:08.735 INFO:teuthology.orchestra.run.trial162.stdout: Installing : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:08.809 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:08.866 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:08.867 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:08.867 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:27:08.867 INFO:teuthology.orchestra.run.trial162.stdout: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:27:08.867 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:08.867 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:08.895 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install rbd-mirror 2026-02-13T19:27:09.195 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:16 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:09.278 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:09.278 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:09.278 INFO:teuthology.orchestra.run.trial162.stdout: Package Arch Version Repo Size 2026-02-13T19:27:09.278 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:09.278 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout: rbd-mirror x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.9 M 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout:Install 1 Package 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 2.9 M 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 11 M 2026-02-13T19:27:09.279 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:27:09.426 INFO:teuthology.orchestra.run.trial162.stdout:rbd-mirror-20.3.0-5230.gafefaa40.el9.x86_64.rpm 19 MB/s | 2.9 MB 00:00 2026-02-13T19:27:09.427 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:27:09.427 INFO:teuthology.orchestra.run.trial162.stdout:Total 19 MB/s | 2.9 MB 00:00 2026-02-13T19:27:09.427 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:27:09.433 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:27:09.433 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:27:09.471 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:27:09.472 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:27:09.646 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:27:09.648 INFO:teuthology.orchestra.run.trial162.stdout: Installing : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:09.671 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:09.671 INFO:teuthology.orchestra.run.trial162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:27:09.671 INFO:teuthology.orchestra.run.trial162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T19:27:09.671 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T19:27:09.671 INFO:teuthology.orchestra.run.trial162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T19:27:09.672 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:09.961 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:09.961 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:09.961 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:27:09.961 INFO:teuthology.orchestra.run.trial162.stdout: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:27:09.961 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:09.961 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:09.989 DEBUG:teuthology.orchestra.run.trial162:> sudo yum -y install rbd-nbd 2026-02-13T19:27:10.292 INFO:teuthology.orchestra.run.trial162.stdout:Last metadata expiration check: 0:01:17 ago on Fri 13 Feb 2026 07:25:53 PM UTC. 2026-02-13T19:27:10.378 INFO:teuthology.orchestra.run.trial162.stdout:Dependencies resolved. 2026-02-13T19:27:10.378 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:10.378 INFO:teuthology.orchestra.run.trial162.stdout: Package Architecture Version Repository Size 2026-02-13T19:27:10.378 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:10.378 INFO:teuthology.orchestra.run.trial162.stdout:Installing: 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout: rbd-nbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 172 k 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout:Transaction Summary 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout:================================================================================ 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout:Install 1 Package 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout:Total download size: 172 k 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout:Installed size: 462 k 2026-02-13T19:27:10.379 INFO:teuthology.orchestra.run.trial162.stdout:Downloading Packages: 2026-02-13T19:27:10.464 INFO:teuthology.orchestra.run.trial162.stdout:rbd-nbd-20.3.0-5230.gafefaa40.el9.x86_64.rpm 2.0 MB/s | 172 kB 00:00 2026-02-13T19:27:10.465 INFO:teuthology.orchestra.run.trial162.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:27:10.465 INFO:teuthology.orchestra.run.trial162.stdout:Total 1.9 MB/s | 172 kB 00:00 2026-02-13T19:27:10.465 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction check 2026-02-13T19:27:10.470 INFO:teuthology.orchestra.run.trial162.stdout:Transaction check succeeded. 2026-02-13T19:27:10.470 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction test 2026-02-13T19:27:10.502 INFO:teuthology.orchestra.run.trial162.stdout:Transaction test succeeded. 2026-02-13T19:27:10.502 INFO:teuthology.orchestra.run.trial162.stdout:Running transaction 2026-02-13T19:27:10.557 INFO:teuthology.orchestra.run.trial162.stdout: Preparing : 1/1 2026-02-13T19:27:10.574 INFO:teuthology.orchestra.run.trial162.stdout: Installing : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:10.651 INFO:teuthology.orchestra.run.trial162.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:10.708 INFO:teuthology.orchestra.run.trial162.stdout: Verifying : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:27:10.708 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:10.708 INFO:teuthology.orchestra.run.trial162.stdout:Installed: 2026-02-13T19:27:10.708 INFO:teuthology.orchestra.run.trial162.stdout: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:27:10.708 INFO:teuthology.orchestra.run.trial162.stdout: 2026-02-13T19:27:10.708 INFO:teuthology.orchestra.run.trial162.stdout:Complete! 2026-02-13T19:27:10.740 DEBUG:teuthology.parallel:result is None 2026-02-13T19:27:10.741 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:27:10.909 DEBUG:teuthology.orchestra.run.trial162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T19:27:10.930 INFO:teuthology.orchestra.run.trial162.stdout:20.3.0-5230.gafefaa40.el9 2026-02-13T19:27:10.931 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5230.gafefaa40.el9 2026-02-13T19:27:10.931 INFO:teuthology.task.install:The correct ceph version 20.3.0-5230.gafefaa40 is installed. 2026-02-13T19:27:10.933 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-13T19:27:10.933 DEBUG:teuthology.orchestra.run.trial162:> set -ex 2026-02-13T19:27:10.933 DEBUG:teuthology.orchestra.run.trial162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T19:27:10.995 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-13T19:27:10.995 DEBUG:teuthology.orchestra.run.trial162:> set -ex 2026-02-13T19:27:10.995 DEBUG:teuthology.orchestra.run.trial162:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T19:27:11.059 DEBUG:teuthology.orchestra.run.trial162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T19:27:11.122 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-13T19:27:11.122 DEBUG:teuthology.orchestra.run.trial162:> set -ex 2026-02-13T19:27:11.122 DEBUG:teuthology.orchestra.run.trial162:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T19:27:11.185 DEBUG:teuthology.orchestra.run.trial162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T19:27:11.250 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-13T19:27:11.250 DEBUG:teuthology.orchestra.run.trial162:> set -ex 2026-02-13T19:27:11.250 DEBUG:teuthology.orchestra.run.trial162:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T19:27:11.314 DEBUG:teuthology.orchestra.run.trial162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T19:27:11.378 INFO:teuthology.run_tasks:Running task workunit... 2026-02-13T19:27:11.404 INFO:tasks.workunit:Pulling workunits from ref afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:27:11.404 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-13T19:27:11.404 INFO:tasks.workunit:timeout=3h 2026-02-13T19:27:11.405 INFO:tasks.workunit:cleanup=True 2026-02-13T19:27:11.405 DEBUG:teuthology.orchestra.run.trial162:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-13T19:27:11.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T19:27:11.434 INFO:teuthology.orchestra.run.trial162.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-13T19:27:11.435 DEBUG:teuthology.orchestra.run.trial162:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-13T19:27:11.490 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-13T19:27:11.490 DEBUG:teuthology.orchestra.run.trial162:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-13T19:27:11.549 DEBUG:teuthology.orchestra.run.trial162:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:27:11.611 INFO:tasks.workunit.client.0.trial162.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-13T19:27:31.621 INFO:tasks.workunit.client.0.trial162.stderr:Note: switching to 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'. 2026-02-13T19:27:31.621 INFO:tasks.workunit.client.0.trial162.stderr: 2026-02-13T19:27:31.621 INFO:tasks.workunit.client.0.trial162.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-13T19:27:31.621 INFO:tasks.workunit.client.0.trial162.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr:state without impacting any branches by switching back to a branch. 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr: 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr: 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr: git switch -c 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr: 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr:Or undo this operation with: 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr: 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr: git switch - 2026-02-13T19:27:31.622 INFO:tasks.workunit.client.0.trial162.stderr: 2026-02-13T19:27:31.623 INFO:tasks.workunit.client.0.trial162.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-13T19:27:31.623 INFO:tasks.workunit.client.0.trial162.stderr: 2026-02-13T19:27:31.623 INFO:tasks.workunit.client.0.trial162.stderr:HEAD is now at afefaa40503 Merge branch 'block_non_4k_chunks' of https://github.com/aainscow/ceph into wip-yuri4-testing-2026-02-11-2126 2026-02-13T19:27:31.628 DEBUG:teuthology.orchestra.run.trial162:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-13T19:27:31.686 DEBUG:teuthology.orchestra.run.trial162:> set -ex 2026-02-13T19:27:31.686 DEBUG:teuthology.orchestra.run.trial162:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-13T19:27:31.743 INFO:tasks.workunit:Running workunits matching misc on client.0... 2026-02-13T19:27:31.744 INFO:tasks.workunit:Running workunit misc/mclock-config.sh... 2026-02-13T19:27:31.744 DEBUG:teuthology.orchestra.run.trial162:workunit test misc/mclock-config.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh 2026-02-13T19:27:31.806 INFO:tasks.workunit.client.0.trial162.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:27:31.809 INFO:tasks.workunit.client.0.trial162.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-13T19:27:31.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/mclock-config 2026-02-13T19:27:31.811 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:21: run: local dir=td/mclock-config 2026-02-13T19:27:31.811 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:22: run: shift 2026-02-13T19:27:31.811 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: export CEPH_MON=127.0.0.1:7124 2026-02-13T19:27:31.811 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: CEPH_MON=127.0.0.1:7124 2026-02-13T19:27:31.811 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:25: run: export CEPH_ARGS 2026-02-13T19:27:31.811 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: uuidgen 2026-02-13T19:27:31.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: CEPH_ARGS+='--fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none ' 2026-02-13T19:27:31.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-13T19:27:31.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:28: run: CEPH_ARGS+='--debug-mclock 20 ' 2026-02-13T19:27:31.812 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: set 2026-02-13T19:27:31.812 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: local 'funcs=TEST_backfill_limit_adjustment_mclock 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:TEST_profile_builtin_to_custom 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:TEST_profile_custom_to_builtin 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:TEST_recovery_limit_adjustment_mclock' 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T19:27:31.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:31.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:31.816 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:31.816 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:31.816 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:31.816 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:31.817 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:31.817 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:31.817 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:31.818 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:31.818 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:31.818 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:31.818 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:31.819 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:31.820 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:31.820 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:31.821 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:31.821 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:31.821 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:31.821 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:31.821 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:31.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:31.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T19:27:31.823 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:27:31.823 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:31.823 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:31.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35773 2026-02-13T19:27:31.824 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_backfill_limit_adjustment_mclock td/mclock-config 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: setup td/mclock-config 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:31.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:31.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:31.827 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:31.827 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:31.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:31.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:31.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:31.828 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:31.828 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:31.828 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:31.828 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:31.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:31.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:31.829 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:31.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:31.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:31.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:31.830 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:31.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:31.832 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:31.832 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:31.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:31.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:31.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:31.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:31.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:31.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:31.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T19:27:31.835 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:27:31.835 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:31.835 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:31.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35773 2026-02-13T19:27:31.836 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:27:31.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:27:31.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:27:31.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T19:27:31.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:239: TEST_backfill_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-13T19:27:31.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T19:27:31.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:27:31.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:27:31.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:27:31.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T19:27:31.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T19:27:31.865 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:27:31.865 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:31.866 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:31.866 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:31.866 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:31.866 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:31.866 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:31.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:27:31.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:27:31.891 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:31.892 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:31.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:31.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get fsid 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:31.936 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:31.937 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:31.937 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:31.937 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:31.937 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:31.937 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:31.937 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get mon_host 2026-02-13T19:27:31.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:240: TEST_backfill_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-13T19:27:31.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T19:27:31.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:27:31.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:27:31.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:27:31.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T19:27:31.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:27:32.077 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:27:32.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:32.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:32.078 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:32.078 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:32.078 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:32.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:32.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:32.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:32.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:242: TEST_backfill_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T19:27:32.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T19:27:32.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:27:32.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:32.093 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T19:27:32.094 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:27:32.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d53f2056-ad13-456e-aa06-e1989d86073c 2026-02-13T19:27:32.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d53f2056-ad13-456e-aa06-e1989d86073c' 2026-02-13T19:27:32.095 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 d53f2056-ad13-456e-aa06-e1989d86073c 2026-02-13T19:27:32.095 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:27:32.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAke49pWqwvBhAAtu+ZUyj1kjLmO3rdmmb6xQ== 2026-02-13T19:27:32.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAke49pWqwvBhAAtu+ZUyj1kjLmO3rdmmb6xQ=="}' 2026-02-13T19:27:32.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d53f2056-ad13-456e-aa06-e1989d86073c -i td/mclock-config/0/new.json 2026-02-13T19:27:32.205 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:32.211 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T19:27:32.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQAke49pWqwvBhAAtu+ZUyj1kjLmO3rdmmb6xQ== --osd-uuid d53f2056-ad13-456e-aa06-e1989d86073c 2026-02-13T19:27:32.228 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.226+0000 7fa6af219780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:32.229 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.228+0000 7fa6af219780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:32.230 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.229+0000 7fa6af219780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:32.231 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.229+0000 7fa6af219780 -1 bdev(0x2d62c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:27:32.231 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.229+0000 7fa6af219780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T19:27:32.611 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T19:27:32.612 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:27:32.612 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:27:32.613 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:27:32.613 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:27:32.714 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:27:32.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:27:32.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T19:27:32.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:27:32.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:27:32.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:27:32.729 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.728+0000 7fe0017df780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:32.731 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.729+0000 7fe0017df780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:32.731 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.730+0000 7fe0017df780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:32.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:32.908 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:27:32.929 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:32.928+0000 7fe0017df780 -1 Falling back to public interface 2026-02-13T19:27:33.253 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:33.251+0000 7fe0017df780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:27:33.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:27:33.910 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:33.910 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:27:33.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:27:33.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:33.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:34.005 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/541613826,v1:127.0.0.1:6801/541613826] [v2:127.0.0.1:6802/541613826,v1:127.0.0.1:6803/541613826] exists,up d53f2056-ad13-456e-aa06-e1989d86073c 2026-02-13T19:27:34.005 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:27:34.006 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:27:34.006 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:27:34.006 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:34.007 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:34.008 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:34.008 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:27:34.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: local backfills=1 2026-02-13T19:27:34.055 INFO:tasks.workunit.client.0.trial162.stdout:osd_max_backfills: 1 2026-02-13T19:27:34.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: echo 'osd_max_backfills: 1' 2026-02-13T19:27:34.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:251: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-13T19:27:34.157 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:252: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-13T19:27:36.160 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T19:27:36.160 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:36.160 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:36.161 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:36.161 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:36.161 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:36.161 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:36.161 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:36.162 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:36.162 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:36.162 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:27:36.208 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: local max_backfills=1 2026-02-13T19:27:36.208 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-13T19:27:36.208 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-13T19:27:36.208 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:36.209 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T19:27:36.256 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-13T19:27:36.257 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:260: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-13T19:27:36.258 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-13T19:27:36.258 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:36.258 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:36.259 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T19:27:36.304 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-13T19:27:36.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:263: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-13T19:27:36.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:267: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-13T19:27:36.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:268: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-13T19:27:36.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:269: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-13T19:27:38.763 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T19:27:38.763 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:38.764 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:38.764 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:38.764 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:38.764 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:38.764 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:38.764 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:38.764 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:38.765 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:38.765 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:27:38.813 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T19:27:38.813 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:272: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T19:27:38.814 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-13T19:27:38.814 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:38.814 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:38.815 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:38.815 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:38.815 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:38.815 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:38.815 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:38.815 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:38.815 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:38.816 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T19:27:38.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T19:27:38.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:277: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:38.863 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:38.864 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:38.864 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:38.864 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:280: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:284: TEST_backfill_limit_adjustment_mclock: kill_daemons td/mclock-config TERM osd 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:38.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:39.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:39.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:285: TEST_backfill_limit_adjustment_mclock: ceph osd down 0 2026-02-13T19:27:39.239 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 is already down. 2026-02-13T19:27:39.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:286: TEST_backfill_limit_adjustment_mclock: wait_for_osd down 0 2026-02-13T19:27:39.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:39.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-13T19:27:39.477 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/541613826,v1:127.0.0.1:6801/541613826] [v2:127.0.0.1:6802/541613826,v1:127.0.0.1:6803/541613826] exists d53f2056-ad13-456e-aa06-e1989d86073c 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:287: TEST_backfill_limit_adjustment_mclock: activate_osd td/mclock-config 0 --osd-op-queue=mclock_scheduler 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/mclock-config 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:27:39.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/mclock-config/0 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:39.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:27:39.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:27:39.484 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=--osd-op-queue=mclock_scheduler 2026-02-13T19:27:39.484 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/mclock-config/0 2026-02-13T19:27:39.484 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:27:39.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T19:27:39.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler 2026-02-13T19:27:39.485 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/mclock-config/0/whoami 2026-02-13T19:27:39.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T19:27:39.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:27:39.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:27:39.486 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:27:39.494 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:39.492+0000 7f5b3c433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:39.495 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:39.494+0000 7f5b3c433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:39.496 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:39.494+0000 7f5b3c433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:39.640 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:39.638+0000 7f5b3c433780 -1 Falling back to public interface 2026-02-13T19:27:39.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T19:27:39.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:27:39.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:27:39.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:27:39.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:27:39.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:39.718 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:39.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:27:39.719 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:39.719 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:39.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:27:39.959 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:39.957+0000 7f5b3c433780 -1 osd.0 6 log_to_monitors true 2026-02-13T19:27:40.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:27:40.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:40.946 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:27:40.946 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:27:40.946 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:40.946 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:41.173 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 6 last_clean_interval [5,6) [v2:127.0.0.1:6800/1065259834,v1:127.0.0.1:6801/1065259834] [v2:127.0.0.1:6802/1065259834,v1:127.0.0.1:6803/1065259834] exists,up d53f2056-ad13-456e-aa06-e1989d86073c 2026-02-13T19:27:41.176 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:27:41.177 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:27:41.177 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:27:41.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-13T19:27:41.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:41.177 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:41.178 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:41.178 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:41.178 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:41.178 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.178 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.178 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:41.178 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:41.179 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:27:41.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T19:27:41.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:291: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T19:27:41.223 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-13T19:27:41.223 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:41.223 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:41.223 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:41.223 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:41.224 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:41.224 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.224 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.224 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:41.224 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:41.224 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T19:27:41.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T19:27:41.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:296: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: bc 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.268 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:41.269 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:41.269 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok dump_recovery_reservations 2026-02-13T19:27:41.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:299: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:301: TEST_backfill_limit_adjustment_mclock: teardown td/mclock-config 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:41.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:41.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:41.421 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:41.421 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:41.422 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:41.423 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:41.424 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:41.424 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:41.424 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:41.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:41.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:41.426 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:41.426 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:41.427 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:41.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:41.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:41.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:41.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:41.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:41.435 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:41.435 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:41.436 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:41.436 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:41.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:41.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:41.437 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:41.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:41.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:41.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:41.439 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:41.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:41.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:41.441 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:41.442 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:41.442 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.442 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.442 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:41.443 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:41.444 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:41.445 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:41.445 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:41.446 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:41.446 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:41.447 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:41.447 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:41.447 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:41.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:41.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:41.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:41.449 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:41.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:41.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:41.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:41.451 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:41.451 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.451 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:41.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:41.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:41.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T19:27:41.453 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:27:41.453 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.453 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.454 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35773 2026-02-13T19:27:41.455 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:27:41.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:27:41.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:27:41.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T19:27:41.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_builtin_to_custom td/mclock-config 2026-02-13T19:27:41.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:39: TEST_profile_builtin_to_custom: local dir=td/mclock-config 2026-02-13T19:27:41.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:41: TEST_profile_builtin_to_custom: run_mon td/mclock-config a 2026-02-13T19:27:41.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T19:27:41.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:27:41.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:27:41.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:27:41.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T19:27:41.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T19:27:41.472 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:27:41.473 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:41.473 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:41.473 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:41.473 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.473 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.473 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:41.473 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:27:41.497 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:27:41.497 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:27:41.497 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:41.497 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:41.497 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:27:41.497 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:27:41.498 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:41.498 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:41.498 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:41.498 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:41.498 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.498 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.498 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:41.499 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:41.499 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get fsid 2026-02-13T19:27:41.541 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:27:41.541 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:41.542 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:41.542 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:27:41.542 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:27:41.543 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:41.543 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:41.543 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:41.543 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:41.544 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.544 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.544 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:41.544 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:41.544 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get mon_host 2026-02-13T19:27:41.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:42: TEST_profile_builtin_to_custom: run_mgr td/mclock-config x 2026-02-13T19:27:41.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T19:27:41.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:27:41.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:27:41.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:27:41.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T19:27:41.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:27:41.687 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:27:41.688 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:41.688 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:41.688 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:41.688 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.688 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.688 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:41.688 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:41.689 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:43: TEST_profile_builtin_to_custom: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:41.704 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T19:27:41.705 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T19:27:41.706 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:27:41.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=744e1cb6-167b-436c-89af-32d0b1fb4e6e 2026-02-13T19:27:41.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 744e1cb6-167b-436c-89af-32d0b1fb4e6e' 2026-02-13T19:27:41.707 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 744e1cb6-167b-436c-89af-32d0b1fb4e6e 2026-02-13T19:27:41.707 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:27:41.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAte49pRt+2KhAADmTYZa32rIdsoR4O2pqBOQ== 2026-02-13T19:27:41.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAte49pRt+2KhAADmTYZa32rIdsoR4O2pqBOQ=="}' 2026-02-13T19:27:41.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 744e1cb6-167b-436c-89af-32d0b1fb4e6e -i td/mclock-config/0/new.json 2026-02-13T19:27:41.819 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:41.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T19:27:41.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQAte49pRt+2KhAADmTYZa32rIdsoR4O2pqBOQ== --osd-uuid 744e1cb6-167b-436c-89af-32d0b1fb4e6e 2026-02-13T19:27:41.842 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:41.840+0000 7f2869835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:41.843 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:41.842+0000 7f2869835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:41.844 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:41.843+0000 7f2869835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:41.844 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:41.843+0000 7f2869835780 -1 bdev(0x3de8c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:27:41.845 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:41.843+0000 7f2869835780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T19:27:42.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T19:27:42.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:27:42.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:27:42.237 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:27:42.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:27:42.340 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:27:42.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:27:42.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T19:27:42.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:27:42.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:27:42.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:27:42.356 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:42.354+0000 7fb30ba87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:42.357 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:42.356+0000 7fb30ba87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:42.358 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:42.357+0000 7fb30ba87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:42.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:27:42.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:27:42.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:27:42.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:27:42.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:27:42.439 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:42.439 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:42.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:27:42.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:42.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:42.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:27:42.574 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:42.573+0000 7fb30ba87780 -1 Falling back to public interface 2026-02-13T19:27:42.887 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:42.886+0000 7fb30ba87780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:27:43.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:27:43.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:43.536 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:27:43.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:27:43.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:43.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:43.632 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1496295046,v1:127.0.0.1:6801/1496295046] [v2:127.0.0.1:6802/1496295046,v1:127.0.0.1:6803/1496295046] exists,up 744e1cb6-167b-436c-89af-32d0b1fb4e6e 2026-02-13T19:27:43.632 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:27:43.632 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:27:43.632 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:27:43.633 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2026-02-13T19:27:43.734 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: local mclock_profile=balanced 2026-02-13T19:27:43.735 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: test balanced = balanced 2026-02-13T19:27:43.735 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:43.736 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T19:27:43.737 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T19:27:43.783 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: mclock_profile='"high_recovery_ops"' 2026-02-13T19:27:43.784 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: eval echo '"high_recovery_ops"' 2026-02-13T19:27:43.784 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: echo high_recovery_ops 2026-02-13T19:27:43.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: mclock_profile=high_recovery_ops 2026-02-13T19:27:43.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_recovery_ops = high_recovery_ops 2026-02-13T19:27:43.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:57: TEST_profile_builtin_to_custom: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-13T19:27:43.841 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:27:43.841 INFO:tasks.workunit.client.0.trial162.stdout: "success": "" 2026-02-13T19:27:43.841 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:27:43.846 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-13T19:27:43.846 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T19:27:43.846 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:43.846 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:43.847 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:43.847 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:43.847 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:43.847 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:43.847 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T19:27:43.847 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T19:27:43.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: mclock_profile='"custom"' 2026-02-13T19:27:43.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: eval echo '"custom"' 2026-02-13T19:27:43.893 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: echo custom 2026-02-13T19:27:43.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: mclock_profile=custom 2026-02-13T19:27:43.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: test custom = custom 2026-02-13T19:27:43.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-13T19:27:43.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: bc 2026-02-13T19:27:43.894 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T19:27:43.894 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:43.894 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:43.895 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:43.895 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:43.895 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:43.895 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:43.895 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T19:27:43.895 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T19:27:43.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: local client_res=0.000000 2026-02-13T19:27:43.943 INFO:tasks.workunit.client.0.trial162.stdout:client_res = 0.000000 2026-02-13T19:27:43.943 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:69: TEST_profile_builtin_to_custom: echo 'client_res = 0.000000' 2026-02-13T19:27:43.944 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: echo '0.000000 + 0.1' 2026-02-13T19:27:43.944 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: bc -l 2026-02-13T19:27:43.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: local client_res_new=.100000 2026-02-13T19:27:43.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:71: TEST_profile_builtin_to_custom: echo 'client_res_new = .100000' 2026-02-13T19:27:43.944 INFO:tasks.workunit.client.0.trial162.stdout:client_res_new = .100000 2026-02-13T19:27:43.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: ceph config set osd.0 osd_mclock_scheduler_client_res .100000 2026-02-13T19:27:44.175 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T19:27:44.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: local res=0.100000 2026-02-13T19:27:44.403 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-13T19:27:44.403 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: bc -l 2026-02-13T19:27:44.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: bc 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-13T19:27:44.406 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T19:27:44.453 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: res=0.100000 2026-02-13T19:27:44.454 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-13T19:27:44.454 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc -l 2026-02-13T19:27:44.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-13T19:27:44.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:89: TEST_profile_builtin_to_custom: teardown td/mclock-config 2026-02-13T19:27:44.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:44.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:44.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:44.456 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:44.456 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:44.457 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:44.457 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:44.457 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:44.562 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:44.562 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:44.563 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:44.564 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:44.565 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:44.565 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:44.565 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:44.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:44.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:44.566 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:44.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:44.569 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:44.571 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:44.572 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.572 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.572 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:44.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:44.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:44.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:44.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:44.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:44.575 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:44.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:44.576 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:44.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:44.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:44.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:44.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:44.579 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:44.579 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.579 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:44.580 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:44.581 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:44.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:44.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.582 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:44.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:44.584 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:44.584 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:44.585 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:44.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:44.586 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:44.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:44.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:44.587 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:44.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:44.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:44.588 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:44.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:44.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:44.591 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:44.591 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.591 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.591 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:44.592 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:44.592 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:44.592 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T19:27:44.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:27:44.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.593 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35773 2026-02-13T19:27:44.594 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_custom_to_builtin td/mclock-config 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:93: TEST_profile_custom_to_builtin: local dir=td/mclock-config 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:95: TEST_profile_custom_to_builtin: setup td/mclock-config 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T19:27:44.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:44.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:44.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:44.596 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:44.596 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:44.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:44.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:44.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:44.597 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:44.597 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:44.598 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:44.598 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:44.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:44.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:44.599 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:44.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:44.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:44.600 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:44.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:44.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:44.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:44.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:44.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.603 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:44.604 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:44.604 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:44.604 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T19:27:44.605 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:27:44.605 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.605 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.605 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35773 2026-02-13T19:27:44.606 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:27:44.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:27:44.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:27:44.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T19:27:44.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:96: TEST_profile_custom_to_builtin: run_mon td/mclock-config a 2026-02-13T19:27:44.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T19:27:44.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:27:44.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:27:44.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:27:44.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T19:27:44.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T19:27:44.624 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:27:44.624 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:44.624 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:44.624 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:44.624 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.624 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.625 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:44.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:27:44.649 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:27:44.649 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:44.650 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.651 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.651 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:44.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:44.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get fsid 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:44.696 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:44.697 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:44.697 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.697 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.697 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:44.697 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:44.698 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get mon_host 2026-02-13T19:27:44.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:97: TEST_profile_custom_to_builtin: run_mgr td/mclock-config x 2026-02-13T19:27:44.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T19:27:44.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:27:44.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:27:44.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:27:44.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T19:27:44.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:27:44.840 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:27:44.840 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:44.841 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:44.841 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:44.841 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.841 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.841 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:44.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:44.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:98: TEST_profile_custom_to_builtin: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T19:27:44.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:44.860 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:27:44.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:27:44.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:27:44.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:27:44.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T19:27:44.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T19:27:44.862 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:27:44.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1c6c5741-b2f8-4eb0-89a2-f123a2f36fe1 2026-02-13T19:27:44.863 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 1c6c5741-b2f8-4eb0-89a2-f123a2f36fe1 2026-02-13T19:27:44.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1c6c5741-b2f8-4eb0-89a2-f123a2f36fe1' 2026-02-13T19:27:44.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:27:44.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAwe49pdX22MxAAXWA0asWyxPHMjxykjczXVw== 2026-02-13T19:27:44.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAwe49pdX22MxAAXWA0asWyxPHMjxykjczXVw=="}' 2026-02-13T19:27:44.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1c6c5741-b2f8-4eb0-89a2-f123a2f36fe1 -i td/mclock-config/0/new.json 2026-02-13T19:27:44.973 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:44.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T19:27:44.980 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQAwe49pdX22MxAAXWA0asWyxPHMjxykjczXVw== --osd-uuid 1c6c5741-b2f8-4eb0-89a2-f123a2f36fe1 2026-02-13T19:27:44.997 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:44.995+0000 7f8b9120b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:44.998 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:44.997+0000 7f8b9120b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:44.999 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:44.998+0000 7f8b9120b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:44.999 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:44.998+0000 7f8b9120b780 -1 bdev(0x37dac00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:27:45.000 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:44.998+0000 7f8b9120b780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T19:27:45.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T19:27:45.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:27:45.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:27:45.400 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:27:45.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:27:45.503 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:27:45.503 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:27:45.503 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:27:45.503 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:27:45.504 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:27:45.504 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T19:27:45.518 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:45.516+0000 7fd81c84f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:45.519 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:45.518+0000 7fd81c84f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:45.520 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:45.519+0000 7fd81c84f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:45.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:27:45.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:27:45.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:27:45.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:27:45.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:27:45.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:45.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:27:45.603 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:45.603 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:45.603 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:45.697 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:27:45.737 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:45.736+0000 7fd81c84f780 -1 Falling back to public interface 2026-02-13T19:27:46.083 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:46.081+0000 7fd81c84f780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:27:46.698 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:27:46.698 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:46.699 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:27:46.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:27:46.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:46.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:46.798 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/744702963,v1:127.0.0.1:6801/744702963] [v2:127.0.0.1:6802/744702963,v1:127.0.0.1:6803/744702963] exists,up 1c6c5741-b2f8-4eb0-89a2-f123a2f36fe1 2026-02-13T19:27:46.799 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:27:46.799 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:27:46.799 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:27:46.800 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: local def_mclock_profile 2026-02-13T19:27:46.800 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2026-02-13T19:27:46.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: def_mclock_profile=balanced 2026-02-13T19:27:46.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:103: TEST_profile_custom_to_builtin: test balanced = balanced 2026-02-13T19:27:46.901 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-13T19:27:46.901 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T19:27:46.901 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:46.901 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:46.902 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:46.902 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:46.902 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:46.902 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:46.902 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T19:27:46.902 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T19:27:46.950 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: local 'orig_mclock_profile="high_recovery_ops"' 2026-02-13T19:27:46.950 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: eval echo '"high_recovery_ops"' 2026-02-13T19:27:46.950 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: echo high_recovery_ops 2026-02-13T19:27:46.950 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: orig_mclock_profile=high_recovery_ops 2026-02-13T19:27:46.950 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:110: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-13T19:27:46.950 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:113: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-13T19:27:47.007 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:27:47.007 INFO:tasks.workunit.client.0.trial162.stdout: "success": "" 2026-02-13T19:27:47.007 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:47.012 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:47.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T19:27:47.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T19:27:47.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: local 'mclock_profile="custom"' 2026-02-13T19:27:47.061 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: eval echo '"custom"' 2026-02-13T19:27:47.062 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: echo custom 2026-02-13T19:27:47.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: mclock_profile=custom 2026-02-13T19:27:47.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:120: TEST_profile_custom_to_builtin: test custom = custom 2026-02-13T19:27:47.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: local client_res 2026-02-13T19:27:47.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T19:27:47.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: bc 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T19:27:47.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T19:27:47.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: client_res=0.000000 2026-02-13T19:27:47.112 INFO:tasks.workunit.client.0.trial162.stdout:Original client_res for osd.0 = 0.000000 2026-02-13T19:27:47.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.0 = 0.000000' 2026-02-13T19:27:47.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: echo '0.000000 + 0.1' 2026-02-13T19:27:47.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: bc -l 2026-02-13T19:27:47.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: local client_res_new=.100000 2026-02-13T19:27:47.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:131: TEST_profile_custom_to_builtin: echo 'client_res_new = .100000' 2026-02-13T19:27:47.113 INFO:tasks.workunit.client.0.trial162.stdout:client_res_new = .100000 2026-02-13T19:27:47.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_scheduler_client_res .100000 2026-02-13T19:27:47.341 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T19:27:47.571 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-13T19:27:47.572 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-13T19:27:47.572 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: bc -l 2026-02-13T19:27:47.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T19:27:47.574 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: bc 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:47.575 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:47.576 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T19:27:47.576 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T19:27:47.621 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: res=0.100000 2026-02-13T19:27:47.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-13T19:27:47.622 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: bc -l 2026-02-13T19:27:47.623 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T19:27:47.623 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:154: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile high_recovery_ops 2026-02-13T19:27:47.681 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:27:47.681 INFO:tasks.workunit.client.0.trial162.stdout: "success": "" 2026-02-13T19:27:47.681 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:27:47.687 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-13T19:27:47.688 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T19:27:47.688 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:47.688 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:47.689 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:47.689 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:47.689 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:47.689 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:47.689 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T19:27:47.689 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_profile 2026-02-13T19:27:47.735 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: eval 'mclock_profile="high_recovery_ops"' 2026-02-13T19:27:47.736 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: mclock_profile=high_recovery_ops 2026-02-13T19:27:47.736 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-13T19:27:47.736 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T19:27:47.966 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-13T19:27:47.967 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-13T19:27:47.967 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: bc -l 2026-02-13T19:27:47.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T19:27:47.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T19:27:47.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: bc 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T19:27:47.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T19:27:48.016 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: res=0.100000 2026-02-13T19:27:48.016 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-13T19:27:48.017 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc -l 2026-02-13T19:27:48.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T19:27:48.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:178: TEST_profile_custom_to_builtin: ceph config rm osd osd_mclock_scheduler_client_res 2026-02-13T19:27:48.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:180: TEST_profile_custom_to_builtin: sleep 5 2026-02-13T19:27:53.269 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-13T19:27:53.499 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: res=0.000000 2026-02-13T19:27:53.499 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: echo '0.000000 != 0.0' 2026-02-13T19:27:53.499 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc -l 2026-02-13T19:27:53.500 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T19:27:53.501 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-13T19:27:53.501 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: bc 2026-02-13T19:27:53.501 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-13T19:27:53.502 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:53.502 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:53.502 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:53.503 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.503 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.503 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:53.503 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-13T19:27:53.503 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-13T19:27:53.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: res=0.000000 2026-02-13T19:27:53.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: echo '0.000000 != 0.000000' 2026-02-13T19:27:53.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: bc -l 2026-02-13T19:27:53.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-13T19:27:53.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:198: TEST_profile_custom_to_builtin: teardown td/mclock-config 2026-02-13T19:27:53.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:53.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:53.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:53.551 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:53.551 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:53.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:53.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:53.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:53.657 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:53.658 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:53.658 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:53.659 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:53.660 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:53.660 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:53.661 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:53.662 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.662 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:53.662 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:53.662 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.663 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:53.664 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:53.664 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:53.667 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:53.667 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.667 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.667 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:53.669 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:53.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:53.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:53.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:53.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:53.671 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:53.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:53.672 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:53.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:53.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:53.673 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:53.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:53.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:53.675 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:53.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:53.677 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:53.678 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:53.678 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.678 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.678 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:53.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:53.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:53.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:53.681 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:53.682 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:53.682 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:53.683 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:53.683 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:53.683 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:53.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:53.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:53.684 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.685 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:53.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:53.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:53.687 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:53.687 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.687 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:53.688 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:53.688 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:53.688 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T19:27:53.689 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:27:53.689 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.689 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.689 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35773 2026-02-13T19:27:53.690 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_recovery_limit_adjustment_mclock td/mclock-config 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: setup td/mclock-config 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:27:53.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:27:53.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:27:53.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:27:53.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:27:53.693 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:27:53.694 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:27:53.694 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:27:53.695 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:27:53.695 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:27:53.695 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:27:53.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:27:53.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:27:53.697 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:27:53.697 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:27:53.698 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:27:53.698 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:27:53.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:27:53.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.699 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:27:53.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:27:53.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:27:53.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-13T19:27:53.701 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:27:53.702 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.702 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.702 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35773 2026-02-13T19:27:53.703 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:27:53.703 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:205: TEST_recovery_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-13T19:27:53.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-13T19:27:53.721 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:27:53.721 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:53.721 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:53.721 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:53.721 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.722 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.722 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:53.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:27:53.746 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:27:53.747 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.748 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:53.749 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:53.749 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get fsid 2026-02-13T19:27:53.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:27:53.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:27:53.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:27:53.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-mon.a.asok 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:27:53.792 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35773/ceph-mon.a.asok config get mon_host 2026-02-13T19:27:53.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:206: TEST_recovery_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-13T19:27:53.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-13T19:27:53.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:27:53.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:27:53.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:27:53.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-13T19:27:53.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:27:53.934 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:27:53.934 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:53.935 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:53.935 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:53.935 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.935 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.935 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:53.935 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:53.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:208: TEST_recovery_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:27:53.952 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-13T19:27:53.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:27:53.954 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=06e5de29-2e00-4973-8004-7b4fdac8f2ef 2026-02-13T19:27:53.954 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 06e5de29-2e00-4973-8004-7b4fdac8f2ef 2026-02-13T19:27:53.954 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 06e5de29-2e00-4973-8004-7b4fdac8f2ef' 2026-02-13T19:27:53.954 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:27:53.963 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA5e49pWexnORAALLrfPrXctC8L9pk0OfJNJw== 2026-02-13T19:27:53.963 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA5e49pWexnORAALLrfPrXctC8L9pk0OfJNJw=="}' 2026-02-13T19:27:53.963 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 06e5de29-2e00-4973-8004-7b4fdac8f2ef -i td/mclock-config/0/new.json 2026-02-13T19:27:54.065 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:54.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-13T19:27:54.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQA5e49pWexnORAALLrfPrXctC8L9pk0OfJNJw== --osd-uuid 06e5de29-2e00-4973-8004-7b4fdac8f2ef 2026-02-13T19:27:54.089 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.087+0000 7f25a342d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:54.090 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.089+0000 7f25a342d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:54.092 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.090+0000 7f25a342d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:54.092 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.090+0000 7f25a342d780 -1 bdev(0x3b84c00 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:27:54.092 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.090+0000 7f25a342d780 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-13T19:27:54.516 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-13T19:27:54.516 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:27:54.516 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:27:54.517 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:27:54.517 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:27:54.618 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:27:54.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:27:54.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=7e33bace-ef38-4290-b9b2-e803f4349af7 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.35773/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T19:27:54.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:27:54.620 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:27:54.620 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:27:54.633 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.632+0000 7faae9c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:54.635 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.633+0000 7faae9c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:54.636 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.634+0000 7faae9c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:27:54.717 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:27:54.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:54.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:54.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:27:54.846 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:54.844+0000 7faae9c2c780 -1 Falling back to public interface 2026-02-13T19:27:55.170 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:27:55.168+0000 7faae9c2c780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:27:55.816 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:27:55.816 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:55.816 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:27:55.817 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:27:55.817 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:55.817 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:55.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:27:56.916 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:27:56.916 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:27:56.916 INFO:tasks.workunit.client.0.trial162.stdout:2 2026-02-13T19:27:56.916 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-13T19:27:56.917 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:27:56.917 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:27:57.144 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2722380850,v1:127.0.0.1:6801/2722380850] [v2:127.0.0.1:6802/2722380850,v1:127.0.0.1:6803/2722380850] exists,up 06e5de29-2e00-4973-8004-7b4fdac8f2ef 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:57.146 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:57.147 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:57.147 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-13T19:27:57.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: local 'recoveries={"osd_recovery_max_active":"0"}' 2026-02-13T19:27:57.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: echo '{"osd_recovery_max_active":"0"}' 2026-02-13T19:27:57.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: grep --quiet osd_recovery_max_active 2026-02-13T19:27:57.197 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:217: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-13T19:27:57.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:218: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-13T19:27:59.435 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:27:59.435 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:27:59.435 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:27:59.435 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:27:59.435 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:27:59.435 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:27:59.436 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:27:59.436 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:27:59.436 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-13T19:27:59.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: local 'max_recoveries={"osd_recovery_max_active":"0"}' 2026-02-13T19:27:59.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"0"}' = '{"osd_recovery_max_active":"0"}' 2026-02-13T19:27:59.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:225: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-13T19:27:59.709 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:226: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-13T19:27:59.941 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:227: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-13T19:28:01.943 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-13T19:28:01.943 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:28:01.943 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:28:01.944 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:28:01.944 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:01.944 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:28:01.944 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35773/ceph-osd.0.asok 2026-02-13T19:28:01.944 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-13T19:28:01.944 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.35773/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-13T19:28:01.993 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: max_recoveries='{"osd_recovery_max_active":"10"}' 2026-02-13T19:28:01.993 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:230: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"10"}' = '{"osd_recovery_max_active":"10"}' 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:232: TEST_recovery_limit_adjustment_mclock: teardown td/mclock-config 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:28:01.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:28:02.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:28:02.101 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:28:02.101 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:28:02.102 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:28:02.103 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:28:02.103 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:28:02.103 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:28:02.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:28:02.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:28:02.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.105 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:28:02.106 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:28:02.106 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:28:02.109 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:28:02.110 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.110 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:28:02.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:28:02.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:28:02.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:28:02.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:28:02.113 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:28:02.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:28:02.114 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:28:02.115 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:28:02.115 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:28:02.115 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:28:02.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:28:02.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:28:02.116 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.117 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:28:02.118 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:28:02.118 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:28:02.119 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:28:02.119 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.119 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:28:02.119 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:28:02.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:28:02.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:28:02.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-13T19:28:02.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/mclock-config 0 2026-02-13T19:28:02.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-13T19:28:02.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T19:28:02.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-13T19:28:02.122 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:28:02.122 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:28:02.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:28:02.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:28:02.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:28:02.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:28:02.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:28:02.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:28:02.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:28:02.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:28:02.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:28:02.125 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:28:02.125 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.126 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:28:02.126 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:28:02.126 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.127 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:28:02.127 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T19:28:02.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-13T19:28:02.129 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:28:02.129 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.129 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35773 2026-02-13T19:28:02.129 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35773 2026-02-13T19:28:02.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:28:02.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:28:02.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-13T19:28:02.131 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T19:28:02.131 DEBUG:teuthology.orchestra.run.trial162:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T19:28:02.197 INFO:tasks.workunit:Running workunit misc/network-ping.sh... 2026-02-13T19:28:02.197 DEBUG:teuthology.orchestra.run.trial162:workunit test misc/network-ping.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh 2026-02-13T19:28:02.259 INFO:tasks.workunit.client.0.trial162.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:28:02.264 INFO:tasks.workunit.client.0.trial162.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:28:02.264 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:28:02.264 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:28:02.264 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-13T19:28:02.264 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-13T19:28:02.264 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/network-ping 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:6: run: local dir=td/network-ping 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:7: run: shift 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: export CEPH_MON=127.0.0.1:7146 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: CEPH_MON=127.0.0.1:7146 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:10: run: export CEPH_ARGS 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: uuidgen 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: CEPH_ARGS+='--fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none ' 2026-02-13T19:28:02.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:12: run: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-13T19:28:02.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:13: run: CEPH_ARGS+='--debug_disable_randomized_ping=true ' 2026-02-13T19:28:02.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:14: run: CEPH_ARGS+='--debug_heartbeat_testing_span=5 ' 2026-02-13T19:28:02.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:15: run: CEPH_ARGS+='--osd_heartbeat_interval=1 ' 2026-02-13T19:28:02.266 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: set 2026-02-13T19:28:02.266 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T19:28:02.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: local 'funcs=TEST_network_ping_test1 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:TEST_network_ping_test2' 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:28:02.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:28:02.269 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:28:02.269 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:28:02.269 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:28:02.270 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:28:02.270 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:28:02.270 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:28:02.270 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:28:02.270 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:28:02.271 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:28:02.271 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.271 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:28:02.272 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:28:02.272 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:28:02.272 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:28:02.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:28:02.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T19:28:02.274 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:28:02.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:02.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46031 2026-02-13T19:28:02.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:28:02.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:28:02.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-13T19:28:02.276 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:28:02.277 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.277 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:02.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.46031 2026-02-13T19:28:02.279 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:28:02.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:28:02.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:28:02.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:28:02.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-13T19:28:02.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test1 td/network-ping 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:25: TEST_network_ping_test1: local dir=td/network-ping 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:27: TEST_network_ping_test1: run_mon td/network-ping a 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-13T19:28:02.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-13T19:28:02.298 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:28:02.298 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:28:02.298 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:28:02.298 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:28:02.299 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.299 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:02.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:02.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:28:02.322 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:28:02.324 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:28:02.324 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:28:02.324 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mon.a.asok 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:28:02.325 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46031/ceph-mon.a.asok config get fsid 2026-02-13T19:28:02.366 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:02.367 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mon.a.asok 2026-02-13T19:28:02.368 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:28:02.368 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46031/ceph-mon.a.asok config get mon_host 2026-02-13T19:28:02.410 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:28: TEST_network_ping_test1: run_mgr td/network-ping x 2026-02-13T19:28:02.410 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-13T19:28:02.410 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:28:02.411 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:28:02.411 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:28:02.411 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-13T19:28:02.411 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:28:02.507 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:28:02.507 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:28:02.507 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:28:02.507 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:28:02.508 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.508 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:02.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:02.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:28:02.508 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:28:02.527 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:29: TEST_network_ping_test1: run_osd td/network-ping 0 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:28:02.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:02.529 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:28:02.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:28:02.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:28:02.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:28:02.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-13T19:28:02.531 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:28:02.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8d751bb8-5a78-4c7d-b911-a76875a94cd7 2026-02-13T19:28:02.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8d751bb8-5a78-4c7d-b911-a76875a94cd7' 2026-02-13T19:28:02.531 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 8d751bb8-5a78-4c7d-b911-a76875a94cd7 2026-02-13T19:28:02.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:28:02.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBCe49pfaz1HxAAPf7Ix460a6k/WsNHPLIerQ== 2026-02-13T19:28:02.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBCe49pfaz1HxAAPf7Ix460a6k/WsNHPLIerQ=="}' 2026-02-13T19:28:02.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8d751bb8-5a78-4c7d-b911-a76875a94cd7 -i td/network-ping/0/new.json 2026-02-13T19:28:02.640 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:28:02.646 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-13T19:28:02.647 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBCe49pfaz1HxAAPf7Ix460a6k/WsNHPLIerQ== --osd-uuid 8d751bb8-5a78-4c7d-b911-a76875a94cd7 2026-02-13T19:28:02.664 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:02.662+0000 7f7d53412780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:02.665 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:02.663+0000 7f7d53412780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:02.666 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:02.664+0000 7f7d53412780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:02.666 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:02.664+0000 7f7d53412780 -1 bdev(0x3d5cc00 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:28:02.666 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:02.664+0000 7f7d53412780 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-13T19:28:03.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-13T19:28:03.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:28:03.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:28:03.086 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:28:03.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:28:03.192 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:28:03.192 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:28:03.193 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:28:03.193 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:28:03.193 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:28:03.193 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:28:03.207 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:03.206+0000 7f2175e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:03.209 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:03.207+0000 7f2175e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:03.210 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:03.208+0000 7f2175e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:28:03.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:28:03.390 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:28:03.416 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:03.415+0000 7f2175e0f780 -1 Falling back to public interface 2026-02-13T19:28:03.821 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:03.819+0000 7f2175e0f780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:28:04.391 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:28:04.391 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:28:04.391 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:28:04.391 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:28:04.391 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:28:04.391 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:28:04.487 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1417915374,v1:127.0.0.1:6801/1417915374] [v2:127.0.0.1:6802/1417915374,v1:127.0.0.1:6803/1417915374] exists,up 8d751bb8-5a78-4c7d-b911-a76875a94cd7 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:30: TEST_network_ping_test1: run_osd td/network-ping 1 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:28:04.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:28:04.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-13T19:28:04.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:28:04.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cbf28b3e-4133-4e13-ae61-bac93ccf8a67 2026-02-13T19:28:04.491 INFO:tasks.workunit.client.0.trial162.stdout:add osd1 cbf28b3e-4133-4e13-ae61-bac93ccf8a67 2026-02-13T19:28:04.491 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 cbf28b3e-4133-4e13-ae61-bac93ccf8a67' 2026-02-13T19:28:04.491 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:28:04.501 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEe49pggfZHRAA1zidTU89dla/RMC5wdaq1g== 2026-02-13T19:28:04.501 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEe49pggfZHRAA1zidTU89dla/RMC5wdaq1g=="}' 2026-02-13T19:28:04.501 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cbf28b3e-4133-4e13-ae61-bac93ccf8a67 -i td/network-ping/1/new.json 2026-02-13T19:28:04.605 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:28:04.610 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-13T19:28:04.611 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEe49pggfZHRAA1zidTU89dla/RMC5wdaq1g== --osd-uuid cbf28b3e-4133-4e13-ae61-bac93ccf8a67 2026-02-13T19:28:04.628 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:04.626+0000 7ff146e12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:04.629 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:04.628+0000 7ff146e12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:04.630 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:04.629+0000 7ff146e12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:04.631 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:04.629+0000 7ff146e12780 -1 bdev(0x2a5ec00 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:28:04.631 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:04.629+0000 7ff146e12780 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-13T19:28:04.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-13T19:28:04.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:28:04.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:28:04.963 INFO:tasks.workunit.client.0.trial162.stdout:adding osd1 key to auth repository 2026-02-13T19:28:04.963 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:28:05.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:28:05.278 INFO:tasks.workunit.client.0.trial162.stdout:start osd.1 2026-02-13T19:28:05.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:28:05.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:28:05.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:28:05.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:28:05.292 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:05.291+0000 7f271d00f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:05.293 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:05.292+0000 7f271d00f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:05.294 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:05.293+0000 7f271d00f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:05.413 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:05.412+0000 7f271d00f780 -1 Falling back to public interface 2026-02-13T19:28:05.510 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:28:05.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:28:05.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:28:05.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:28:05.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:28:05.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:28:05.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:28:05.511 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:28:05.512 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:28:05.512 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:28:05.725 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:05.723+0000 7f271d00f780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:28:05.736 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:28:06.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:28:06.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:28:06.738 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:28:06.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:28:06.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:28:06.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:28:06.966 INFO:tasks.workunit.client.0.trial162.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1689375985,v1:127.0.0.1:6811/1689375985] [v2:127.0.0.1:6812/1689375985,v1:127.0.0.1:6813/1689375985] exists,up cbf28b3e-4133-4e13-ae61-bac93ccf8a67 2026-02-13T19:28:06.970 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:28:06.970 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:31: TEST_network_ping_test1: run_osd td/network-ping 2 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:28:06.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:06.972 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:28:06.973 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-13T19:28:06.974 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:28:06.974 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=791c8019-7680-4924-beaa-86eb5014d26e 2026-02-13T19:28:06.974 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 791c8019-7680-4924-beaa-86eb5014d26e' 2026-02-13T19:28:06.974 INFO:tasks.workunit.client.0.trial162.stdout:add osd2 791c8019-7680-4924-beaa-86eb5014d26e 2026-02-13T19:28:06.974 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:28:06.980 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBGe49pj8FfOhAAGMy+U629zvEHUBrmcNXyFA== 2026-02-13T19:28:06.980 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBGe49pj8FfOhAAGMy+U629zvEHUBrmcNXyFA=="}' 2026-02-13T19:28:06.980 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 791c8019-7680-4924-beaa-86eb5014d26e -i td/network-ping/2/new.json 2026-02-13T19:28:07.212 INFO:tasks.workunit.client.0.trial162.stdout:2 2026-02-13T19:28:07.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-13T19:28:07.222 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBGe49pj8FfOhAAGMy+U629zvEHUBrmcNXyFA== --osd-uuid 791c8019-7680-4924-beaa-86eb5014d26e 2026-02-13T19:28:07.237 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.236+0000 7f7107c08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:07.239 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.237+0000 7f7107c08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:07.239 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.238+0000 7f7107c08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:07.240 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.238+0000 7f7107c08780 -1 bdev(0x2a1ac00 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:28:07.240 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.238+0000 7f7107c08780 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-13T19:28:07.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-13T19:28:07.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:28:07.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:28:07.552 INFO:tasks.workunit.client.0.trial162.stdout:adding osd2 key to auth repository 2026-02-13T19:28:07.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:28:07.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:28:07.879 INFO:tasks.workunit.client.0.trial162.stdout:start osd.2 2026-02-13T19:28:07.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:28:07.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:28:07.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:28:07.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:28:07.894 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.892+0000 7fcfe4013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:07.895 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.894+0000 7fcfe4013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:07.896 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:07.895+0000 7fcfe4013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:28:08.064 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:08.063+0000 7fcfe4013780 -1 Falling back to public interface 2026-02-13T19:28:08.109 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:28:08.109 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:28:08.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:28:08.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:28:08.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:28:08.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:28:08.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:28:08.110 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:28:08.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:28:08.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:28:08.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:28:08.366 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:28:08.365+0000 7fcfe4013780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:28:09.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:28:09.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:28:09.342 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:28:09.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:28:09.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:28:09.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:28:09.566 INFO:tasks.workunit.client.0.trial162.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/690479804,v1:127.0.0.1:6819/690479804] [v2:127.0.0.1:6820/690479804,v1:127.0.0.1:6821/690479804] exists,up 791c8019-7680-4924-beaa-86eb5014d26e 2026-02-13T19:28:09.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:28:09.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:28:09.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:28:09.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:33: TEST_network_ping_test1: sleep 5 2026-02-13T19:28:14.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:35: TEST_network_ping_test1: create_pool foo 16 2026-02-13T19:28:14.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-13T19:28:14.846 INFO:tasks.workunit.client.0.trial162.stderr:pool 'foo' already exists 2026-02-13T19:28:14.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:28:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:38: TEST_network_ping_test1: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-13T19:28:15.876 INFO:tasks.workunit.client.0.trial162.stdout:hints = 1 2026-02-13T19:28:15.876 INFO:tasks.workunit.client.0.trial162.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-13T19:28:15.876 INFO:tasks.workunit.client.0.trial162.stdout:Object prefix: benchmark_data_trial162_50769 2026-02-13T19:28:15.876 INFO:tasks.workunit.client.0.trial162.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-13T19:28:15.876 INFO:tasks.workunit.client.0.trial162.stdout: 0 0 0 0 0 0 - 0 2026-02-13T19:28:16.876 INFO:tasks.workunit.client.0.trial162.stdout: 1 16 11795 11779 46.0083 46.0117 0.000896456 0.001356 2026-02-13T19:28:17.876 INFO:tasks.workunit.client.0.trial162.stdout: 2 16 23547 23531 45.9559 45.9062 0.0012132 0.0013581 2026-02-13T19:28:18.876 INFO:tasks.workunit.client.0.trial162.stdout: 3 16 35196 35180 45.804 45.5039 0.000874505 0.00136178 2026-02-13T19:28:19.876 INFO:tasks.workunit.client.0.trial162.stdout: 4 16 47153 47137 46.0289 46.707 0.000978949 0.00135628 2026-02-13T19:28:20.877 INFO:tasks.workunit.client.0.trial162.stdout: 5 16 58974 58958 46.0575 46.1758 0.00217914 0.00135559 2026-02-13T19:28:21.876 INFO:tasks.workunit.client.0.trial162.stdout: 6 16 70848 70832 46.1111 46.3828 0.000908428 0.00135397 2026-02-13T19:28:22.877 INFO:tasks.workunit.client.0.trial162.stdout: 7 16 82575 82559 46.0674 45.8086 0.0014286 0.00135524 2026-02-13T19:28:23.877 INFO:tasks.workunit.client.0.trial162.stdout: 8 16 93078 93062 45.437 41.0273 0.00157562 0.00137411 2026-02-13T19:28:24.877 INFO:tasks.workunit.client.0.trial162.stdout: 9 16 103608 103592 44.9584 41.1328 0.00149076 0.00138876 2026-02-13T19:28:25.877 INFO:tasks.workunit.client.0.trial162.stdout: 10 6 113891 113885 44.4829 40.207 0.00115065 0.00140368 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Total time run: 10.0009 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Total writes made: 113891 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Write size: 4096 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Object size: 4096 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Bandwidth (MB/sec): 44.4849 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Stddev Bandwidth: 2.58268 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Max bandwidth (MB/sec): 46.707 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Min bandwidth (MB/sec): 40.207 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Average IOPS: 11388 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Stddev IOPS: 661.167 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Max IOPS: 11957 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Min IOPS: 10293 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Average Latency(s): 0.00140366 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Stddev Latency(s): 0.000729737 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Max latency(s): 0.0142258 2026-02-13T19:28:26.877 INFO:tasks.workunit.client.0.trial162.stdout:Min latency(s): 0.000417986 2026-02-13T19:28:26.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:41: TEST_network_ping_test1: sleep 10 2026-02-13T19:28:36.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:42: TEST_network_ping_test1: flush_pg_stats 2026-02-13T19:28:36.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:28:36.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:28:37.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:28:37.111 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:28:37.111 INFO:tasks.workunit.client.0.trial162.stderr:2' 2026-02-13T19:28:37.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:28:37.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:28:37.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:28:37.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-13T19:28:37.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-13T19:28:37.176 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489' 2026-02-13T19:28:37.176 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:28:37.176 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:28:37.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705673 2026-02-13T19:28:37.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705673 2026-02-13T19:28:37.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489 1-38654705673' 2026-02-13T19:28:37.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:28:37.240 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:28:37.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542152 2026-02-13T19:28:37.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542152 2026-02-13T19:28:37.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489 1-38654705673 2-60129542152' 2026-02-13T19:28:37.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:28:37.305 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836489 2026-02-13T19:28:37.305 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:28:37.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:28:37.307 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-13T19:28:37.307 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:28:37.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836489 2026-02-13T19:28:37.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-13T19:28:37.308 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836489 2026-02-13T19:28:37.308 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:28:37.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-13T19:28:37.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:28:38.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:28:38.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:28:38.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-13T19:28:38.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:28:38.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705673 2026-02-13T19:28:38.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:28:38.946 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:28:38.946 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705673 2026-02-13T19:28:38.946 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:28:38.947 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705673 2026-02-13T19:28:38.948 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-13T19:28:38.948 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 38654705673 2026-02-13T19:28:38.948 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:28:39.245 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705673 -lt 38654705673 2026-02-13T19:28:39.245 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:28:39.246 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542152 2026-02-13T19:28:39.246 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:28:39.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:28:39.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542152 2026-02-13T19:28:39.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:28:39.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542152 2026-02-13T19:28:39.248 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542152 2026-02-13T19:28:39.249 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542152' 2026-02-13T19:28:39.249 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:28:39.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542152 -lt 60129542152 2026-02-13T19:28:39.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:28:39.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T19:28:39.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:28:39.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:28:39.550 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:28:39.550 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:39.550 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:39.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-osd.0.asok 2026-02-13T19:28:39.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:28:39.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-osd.0.asok dump_osd_network 2026-02-13T19:28:39.594 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:28:39.594 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 1000, 2026-02-13T19:28:39.594 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:28:39.594 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:28:39.599 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.599 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:28:39.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:28:39.601 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.602 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: jq .threshold 2026-02-13T19:28:39.603 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T19:28:39.604 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:28:39.604 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T19:28:39.604 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T19:28:39.604 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T19:28:39.605 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:28:39.605 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:39.605 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:39.605 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mgr.x.asok 2026-02-13T19:28:39.605 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:28:39.605 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-mgr.x.asok dump_osd_network 2026-02-13T19:28:39.642 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:28:39.642 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 1000, 2026-02-13T19:28:39.642 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:28:39.642 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:28:39.675 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.675 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:28:39.677 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:28:39.677 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.677 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: jq .threshold 2026-02-13T19:28:39.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T19:28:39.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:28:39.681 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T19:28:39.681 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:28:39.681 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:28:39.681 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:28:39.681 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:39.681 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:39.682 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-osd.0.asok 2026-02-13T19:28:39.682 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:28:39.682 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-osd.0.asok dump_osd_network 0 2026-02-13T19:28:39.725 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:28:39.725 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 0, 2026-02-13T19:28:39.725 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [ 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.106, 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.134, 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.138 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.726 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.114, 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.177, 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.177 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.114, 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.177, 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.177 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.125 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.727 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:37 2026", 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.087, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.135, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.110 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.120, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.348, 2026-02-13T19:28:39.728 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.348 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.120, 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.348, 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.348 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.107 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.729 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.091, 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.124, 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.124 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.099, 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.180, 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.180 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.730 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.099, 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.180, 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.180 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.133 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:37 2026", 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.731 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.103, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.122, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.122 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.140, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.196, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.196 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.140, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.196, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.196 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.732 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.115 2026-02-13T19:28:39.733 INFO:tasks.workunit.client.0.trial162.stdout: } 2026-02-13T19:28:39.733 INFO:tasks.workunit.client.0.trial162.stdout: ] 2026-02-13T19:28:39.733 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:28:39.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:28:39.755 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: test 4 = 4 2026-02-13T19:28:39.756 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.756 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: jq .threshold 2026-02-13T19:28:39.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:28:39.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:28:39.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T19:28:39.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T19:28:39.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T19:28:39.760 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:28:39.760 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:28:39.760 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:28:39.760 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mgr.x.asok 2026-02-13T19:28:39.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:28:39.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-mgr.x.asok dump_osd_network 0 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 0, 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [ 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:36 2026", 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:28:39.798 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.109, 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.142, 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.560 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.090, 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.088, 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.088 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.135, 2026-02-13T19:28:39.799 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.298, 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 3.415 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.135 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:36 2026", 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.089, 2026-02-13T19:28:39.800 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.120, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.524 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.064, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.064, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.064 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.201, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 3.448 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.801 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.110 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:35 2026", 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.112, 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.212, 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.170 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.802 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.096, 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.096, 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.096 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.129, 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 1.450, 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 1.450 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.117 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.803 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:35 2026", 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.094, 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.196, 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.156 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.087, 2026-02-13T19:28:39.804 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.087, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.087 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.104, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 1.431, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 1.431 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.097 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:28:39.805 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.102, 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.182, 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.152 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.075, 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.075, 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.075 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.806 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.119, 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.480, 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.480 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.086 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.807 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.123, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.171, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.159 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.110, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.110 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.136, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.398, 2026-02-13T19:28:39.808 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.398 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.092 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.097, 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.144, 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.124 2026-02-13T19:28:39.809 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.071, 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.071, 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.071 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.162, 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.335, 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.335 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.080 2026-02-13T19:28:39.810 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.092, 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.142, 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.132 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.811 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.067, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.067, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.067 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.375, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.375 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.097 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.812 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.106, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.134, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.138 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.101, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.101, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.101 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.813 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.114, 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.177, 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.177 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.112 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:32 2026", 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.814 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.071, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.137, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.111 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.065, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.065, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.065 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.085, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.348, 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.348 2026-02-13T19:28:39.815 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.082 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:32 2026", 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.091, 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.126, 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.124 2026-02-13T19:28:39.816 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.081, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.081, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.081 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.105, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.196, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.196 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.124 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.817 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:34 2026", 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.091, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.124, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.124 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.083, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.080, 2026-02-13T19:28:39.818 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.080 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.099, 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.180, 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.180 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.093 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: } 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout: ] 2026-02-13T19:28:39.819 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:28:39.820 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.820 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:28:39.821 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: test 12 = 12 2026-02-13T19:28:39.821 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:28:39.821 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: jq .threshold 2026-02-13T19:28:39.821 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:28:39.821 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:61: TEST_network_ping_test1: sleep 20 2026-02-13T19:28:59.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:62: TEST_network_ping_test1: flush_pg_stats 2026-02-13T19:28:59.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:28:59.812 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:29:00.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:29:00.045 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:29:00.045 INFO:tasks.workunit.client.0.trial162.stderr:2' 2026-02-13T19:29:00.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:29:00.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:29:00.045 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:29:00.108 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836496 2026-02-13T19:29:00.109 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836496 2026-02-13T19:29:00.109 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496' 2026-02-13T19:29:00.109 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:29:00.109 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:29:00.174 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705679 2026-02-13T19:29:00.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705679 2026-02-13T19:29:00.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496 1-38654705679' 2026-02-13T19:29:00.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:29:00.175 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:29:00.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542159 2026-02-13T19:29:00.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542159 2026-02-13T19:29:00.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836496 1-38654705679 2-60129542159' 2026-02-13T19:29:00.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:29:00.240 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836496 2026-02-13T19:29:00.240 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:29:00.241 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:29:00.241 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836496 2026-02-13T19:29:00.242 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:29:00.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836496 2026-02-13T19:29:00.243 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-13T19:29:00.243 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836496 2026-02-13T19:29:00.243 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:29:00.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836493 -lt 21474836496 2026-02-13T19:29:00.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:29:01.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:29:01.477 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:29:01.706 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-13T19:29:01.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:29:01.707 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705679 2026-02-13T19:29:01.707 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:29:01.708 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:29:01.709 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705679 2026-02-13T19:29:01.709 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:29:01.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705679 2026-02-13T19:29:01.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-13T19:29:01.711 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 38654705679 2026-02-13T19:29:01.711 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:29:01.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705679 -lt 38654705679 2026-02-13T19:29:01.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:29:01.943 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542159 2026-02-13T19:29:01.943 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:29:01.943 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:29:01.944 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542159 2026-02-13T19:29:01.944 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:29:01.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542159 2026-02-13T19:29:01.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542159' 2026-02-13T19:29:01.946 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542159 2026-02-13T19:29:01.946 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:29:02.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542159 -lt 60129542159 2026-02-13T19:29:02.184 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:02.184 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T19:29:02.184 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:29:02.184 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:29:02.184 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:02.185 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:02.185 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:02.185 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-osd.0.asok 2026-02-13T19:29:02.185 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:02.185 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-osd.0.asok dump_osd_network 2026-02-13T19:29:02.227 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:02.227 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 1000, 2026-02-13T19:29:02.228 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:29:02.228 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:02.232 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.232 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:02.233 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:02.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:02.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T19:29:02.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:02.238 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T19:29:02.238 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T19:29:02.238 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T19:29:02.239 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:02.239 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:02.239 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:02.239 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mgr.x.asok 2026-02-13T19:29:02.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:02.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-mgr.x.asok dump_osd_network 2026-02-13T19:29:02.277 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:02.277 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 1000, 2026-02-13T19:29:02.277 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:29:02.277 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:02.282 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.282 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:02.284 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:02.284 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.285 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:02.286 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T19:29:02.287 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:02.288 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T19:29:02.288 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:29:02.288 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:29:02.288 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:02.288 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:02.288 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:02.288 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-osd.0.asok 2026-02-13T19:29:02.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:02.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-osd.0.asok dump_osd_network 0 2026-02-13T19:29:02.333 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:02.333 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 0, 2026-02-13T19:29:02.333 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [ 2026-02-13T19:29:02.333 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.334 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.334 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.334 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.334 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:02.334 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.334 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.334 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.099, 2026-02-13T19:29:02.335 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.119, 2026-02-13T19:29:02.335 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.131 2026-02-13T19:29:02.335 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.335 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.335 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.118, 2026-02-13T19:29:02.335 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.176, 2026-02-13T19:29:02.336 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.177 2026-02-13T19:29:02.336 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.336 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.336 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.118, 2026-02-13T19:29:02.336 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.176, 2026-02-13T19:29:02.336 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.177 2026-02-13T19:29:02.337 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.337 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.130 2026-02-13T19:29:02.337 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.337 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.337 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.337 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.338 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.338 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:02.338 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.338 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.338 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.092, 2026-02-13T19:29:02.338 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.110, 2026-02-13T19:29:02.339 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.119 2026-02-13T19:29:02.339 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.339 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.339 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:29:02.339 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.182, 2026-02-13T19:29:02.339 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.182 2026-02-13T19:29:02.340 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.340 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.340 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:29:02.340 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.182, 2026-02-13T19:29:02.340 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.182 2026-02-13T19:29:02.340 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.341 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.143 2026-02-13T19:29:02.341 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.341 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.341 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:57 2026", 2026-02-13T19:29:02.341 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.341 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.342 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:02.342 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.342 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.342 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.087, 2026-02-13T19:29:02.342 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.103, 2026-02-13T19:29:02.342 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.117 2026-02-13T19:29:02.342 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.343 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.343 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.100, 2026-02-13T19:29:02.343 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.164, 2026-02-13T19:29:02.343 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.196 2026-02-13T19:29:02.343 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.343 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.343 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.100, 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.164, 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.196 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.115 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:57 2026", 2026-02-13T19:29:02.344 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.345 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.345 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:02.345 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.345 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.345 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.097, 2026-02-13T19:29:02.345 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.101, 2026-02-13T19:29:02.345 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.112 2026-02-13T19:29:02.346 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.346 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.346 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.116, 2026-02-13T19:29:02.346 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.148, 2026-02-13T19:29:02.346 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.348 2026-02-13T19:29:02.346 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.346 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.116, 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.148, 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.348 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.154 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout: } 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout: ] 2026-02-13T19:29:02.347 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:02.349 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: test 4 = 4 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T19:29:02.350 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:02.351 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:02.351 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:02.351 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mgr.x.asok 2026-02-13T19:29:02.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:02.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-mgr.x.asok dump_osd_network 0 2026-02-13T19:29:02.381 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:02.381 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 0, 2026-02-13T19:29:02.381 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [ 2026-02-13T19:29:02.381 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:56 2026", 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.095, 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.116, 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.386 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.382 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.090, 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.090, 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.088 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.107, 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.156, 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 3.415 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.092 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.383 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:56 2026", 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.091, 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.102, 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.359 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.080, 2026-02-13T19:29:02.384 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.064, 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.064 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.113, 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.150, 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 3.448 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.084 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.385 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:55 2026", 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.116, 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.128, 2026-02-13T19:29:02.386 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.165 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.105, 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.096, 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.096 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.134, 2026-02-13T19:29:02.387 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.181, 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 1.450 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.112 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:55 2026", 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.388 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.114, 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.120, 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.154 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.389 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.101, 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.087, 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.087 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.143, 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.178, 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 1.431 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.390 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.102 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.391 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.107, 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.132, 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.152 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.103, 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.092, 2026-02-13T19:29:02.392 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.092 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.113, 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.207, 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.398 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.104 2026-02-13T19:29:02.393 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.098, 2026-02-13T19:29:02.394 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.122, 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.147 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.087, 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.086, 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.075 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.395 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.112, 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.199, 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.480 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.095 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.396 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.099, 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.119, 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.131 2026-02-13T19:29:02.397 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.076, 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.076, 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.076 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.118, 2026-02-13T19:29:02.398 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.176, 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.177 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.076 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:02.399 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.077, 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.109, 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.126 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.400 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.064, 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.064, 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.064 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.095, 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.220, 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.375 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.401 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.064 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.402 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.085, 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.101, 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.120 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.078, 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.068, 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.068 2026-02-13T19:29:02.403 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.092, 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.206, 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.335 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.083 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.404 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:59 2026", 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.092, 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.110, 2026-02-13T19:29:02.405 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.119 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.077, 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.074, 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.074 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.406 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.182, 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.182 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.084 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:57 2026", 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.407 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.087, 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.103, 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.117 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.408 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.076, 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.076, 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.076 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.100, 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.164, 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.196 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.409 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.098 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:28:57 2026", 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.097, 2026-02-13T19:29:02.410 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.101, 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.112 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.083, 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.067, 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.065 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:02.411 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.116, 2026-02-13T19:29:02.412 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.148, 2026-02-13T19:29:02.412 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.348 2026-02-13T19:29:02.412 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:02.412 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.091 2026-02-13T19:29:02.412 INFO:tasks.workunit.client.0.trial162.stdout: } 2026-02-13T19:29:02.412 INFO:tasks.workunit.client.0.trial162.stdout: ] 2026-02-13T19:29:02.412 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:02.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.414 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:02.414 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: test 12 = 12 2026-02-13T19:29:02.414 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:02.414 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:02.414 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:02.414 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:81: TEST_network_ping_test1: sleep 50 2026-02-13T19:29:52.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:82: TEST_network_ping_test1: flush_pg_stats 2026-02-13T19:29:52.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:29:52.392 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:29:52.621 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:29:52.621 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:29:52.621 INFO:tasks.workunit.client.0.trial162.stderr:2' 2026-02-13T19:29:52.621 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:29:52.621 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:29:52.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:29:52.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836508 2026-02-13T19:29:52.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836508 2026-02-13T19:29:52.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508' 2026-02-13T19:29:52.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:29:52.687 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:29:52.749 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705692 2026-02-13T19:29:52.749 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705692 2026-02-13T19:29:52.749 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705692' 2026-02-13T19:29:52.749 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:29:52.749 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:29:52.813 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542171 2026-02-13T19:29:52.813 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542171 2026-02-13T19:29:52.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705692 2-60129542171' 2026-02-13T19:29:52.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:29:52.814 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836508 2026-02-13T19:29:52.814 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:29:52.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:29:52.815 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836508 2026-02-13T19:29:52.815 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:29:52.816 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836508 2026-02-13T19:29:52.816 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-13T19:29:52.817 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836508 2026-02-13T19:29:52.817 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:29:53.050 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836506 -lt 21474836508 2026-02-13T19:29:53.050 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:29:54.050 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:29:54.051 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:29:54.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836506 -lt 21474836508 2026-02-13T19:29:54.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:29:55.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:29:55.277 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:29:55.508 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-13T19:29:55.508 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:29:55.509 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705692 2026-02-13T19:29:55.509 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:29:55.509 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:29:55.510 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705692 2026-02-13T19:29:55.510 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:29:55.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705692 2026-02-13T19:29:55.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705692' 2026-02-13T19:29:55.512 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 38654705692 2026-02-13T19:29:55.512 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:29:55.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705692 -lt 38654705692 2026-02-13T19:29:55.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:29:55.742 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542171 2026-02-13T19:29:55.742 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:29:55.742 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:29:55.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542171 2026-02-13T19:29:55.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:29:55.744 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542171 2026-02-13T19:29:55.745 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542171' 2026-02-13T19:29:55.745 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542171 2026-02-13T19:29:55.745 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:29:55.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542171 -lt 60129542171 2026-02-13T19:29:55.980 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:55.980 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T19:29:55.980 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:29:55.980 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:29:55.980 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:55.980 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:55.981 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:55.981 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-osd.0.asok 2026-02-13T19:29:55.981 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:55.981 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-osd.0.asok dump_osd_network 2026-02-13T19:29:56.024 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:56.025 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 1000, 2026-02-13T19:29:56.025 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:29:56.025 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:56.029 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.030 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:56.031 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:56.032 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.032 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:56.034 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T19:29:56.034 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:56.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T19:29:56.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T19:29:56.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T19:29:56.036 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:56.037 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:56.037 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:56.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mgr.x.asok 2026-02-13T19:29:56.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:56.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-mgr.x.asok dump_osd_network 2026-02-13T19:29:56.074 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:56.074 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 1000, 2026-02-13T19:29:56.074 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:29:56.074 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:56.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:56.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:56.082 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.082 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:56.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: test 1000 = 1000 2026-02-13T19:29:56.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:56.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T19:29:56.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:29:56.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:29:56.085 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:56.085 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:56.085 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:56.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-osd.0.asok 2026-02-13T19:29:56.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:56.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-osd.0.asok dump_osd_network 0 2026-02-13T19:29:56.128 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:56.128 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 0, 2026-02-13T19:29:56.128 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [ 2026-02-13T19:29:56.128 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.128 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:54 2026", 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.130, 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.118, 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.115 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.169, 2026-02-13T19:29:56.129 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.182, 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.182 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.169, 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.182, 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.182 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.176 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:54 2026", 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.130 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.129, 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.109, 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.117 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.208, 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.208, 2026-02-13T19:29:56.131 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.208 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.208, 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.208, 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.208 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.188 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:52 2026", 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.132 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.094, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.084, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.099 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.155, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.155, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.164 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.133 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.155, 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.155, 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.164 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.152 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:52 2026", 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.134 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.089, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.076, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.090 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.184, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.184, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.184 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.184, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.184, 2026-02-13T19:29:56.135 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.184 2026-02-13T19:29:56.136 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.136 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.140 2026-02-13T19:29:56.136 INFO:tasks.workunit.client.0.trial162.stdout: } 2026-02-13T19:29:56.136 INFO:tasks.workunit.client.0.trial162.stdout: ] 2026-02-13T19:29:56.136 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:56.137 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.137 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:56.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: test 4 = 4 2026-02-13T19:29:56.137 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.137 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:56.138 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:56.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mgr.x.asok 2026-02-13T19:29:56.140 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:56.140 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-mgr.x.asok dump_osd_network 0 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 0, 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [ 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:50 2026", 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:56.177 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.108, 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.130, 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.131 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.095, 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.093, 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.093 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.178 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.115, 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.453, 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.453 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.159 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:49 2026", 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.179 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.116, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.119, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.129 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.098, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.085 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.130, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.160, 2026-02-13T19:29:56.180 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.207 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.116 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:49 2026", 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.083, 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.102, 2026-02-13T19:29:56.181 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.123 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.073, 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.073, 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.073 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.107, 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.147, 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.246 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.083 2026-02-13T19:29:56.182 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:50 2026", 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.098, 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.114, 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.119 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.183 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.088, 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.085, 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.084 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.110, 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.446, 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.446 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.128 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:49 2026", 2026-02-13T19:29:56.184 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.114, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.119, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.114 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.098, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.075, 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.074 2026-02-13T19:29:56.185 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.139, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.182, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.182 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.131 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:49 2026", 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 2, 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.186 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.084, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.107, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.117 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.078, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.078, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.076 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.092, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.149, 2026-02-13T19:29:56.187 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.176 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.143 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:51 2026", 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.092, 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.114, 2026-02-13T19:29:56.188 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.116 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.080, 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.080, 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.079 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.101, 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.438, 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.438 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.136 2026-02-13T19:29:56.189 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:49 2026", 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.109, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.107, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.112 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.101, 2026-02-13T19:29:56.190 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.079, 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.064 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.117, 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.142, 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.220 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.125 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:49 2026", 2026-02-13T19:29:56.191 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 2, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.106, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.107, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.107 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.091, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.077, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.068 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.192 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.149, 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.179, 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.206 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.103 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:47 2026", 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "front", 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.193 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.074, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.087, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.099 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.071, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.068, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.068 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.082, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.139, 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.164 2026-02-13T19:29:56.194 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.074 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:51 2026", 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 1, 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 0, 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.079, 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.096, 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.097 2026-02-13T19:29:56.195 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.074, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.070, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.069 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.083, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.428, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.428 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.115 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: { 2026-02-13T19:29:56.196 INFO:tasks.workunit.client.0.trial162.stdout: "last update": "Fri Feb 13 19:29:47 2026", 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "stale": false, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "from osd": 0, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "to osd": 1, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "interface": "back", 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "average": { 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.064, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.074, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.090 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "min": { 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.063, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.060, 2026-02-13T19:29:56.197 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.060 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: "max": { 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: "1min": 0.068, 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: "5min": 0.095, 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: "15min": 0.154 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: }, 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: "last": 0.063 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: } 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout: ] 2026-02-13T19:29:56.198 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:56.200 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: jq '.entries | length' 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: test 12 = 12 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: test 0 = 0 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:56.201 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:56.202 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:56.202 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mgr.x.asok 2026-02-13T19:29:56.202 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:56.202 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-mgr.x.asok dump_osd_network 99 2026-02-13T19:29:56.226 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:56.227 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 99, 2026-02-13T19:29:56.227 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:29:56.227 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:56.231 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.231 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:56.233 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: test 99 = 99 2026-02-13T19:29:56.233 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: tee td/network-ping/json 2026-02-13T19:29:56.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: get_asok_path osd.0 2026-02-13T19:29:56.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:29:56.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:29:56.234 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:56.234 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:56.234 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:56.235 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-osd.0.asok 2026-02-13T19:29:56.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: CEPH_ARGS= 2026-02-13T19:29:56.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.46031/ceph-osd.0.asok dump_osd_network 98 2026-02-13T19:29:56.277 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:29:56.277 INFO:tasks.workunit.client.0.trial162.stdout: "threshold": 98, 2026-02-13T19:29:56.277 INFO:tasks.workunit.client.0.trial162.stdout: "entries": [] 2026-02-13T19:29:56.277 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:29:56.282 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: cat td/network-ping/json 2026-02-13T19:29:56.282 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: jq .threshold 2026-02-13T19:29:56.284 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: test 98 = 98 2026-02-13T19:29:56.284 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:105: TEST_network_ping_test1: rm -f td/network-ping/json 2026-02-13T19:29:56.285 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-13T19:29:56.285 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T19:29:56.285 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:29:56.285 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T19:29:56.286 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:29:56.286 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:29:56.286 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:29:56.286 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:29:56.286 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:29:56.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:29:56.393 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:29:56.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:29:56.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:29:56.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:29:56.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:29:56.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:29:56.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:29:56.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:29:56.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:29:56.396 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:29:56.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:29:56.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:29:56.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T19:29:57.003 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:29:57.003 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.003 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.003 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46031 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:29:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T19:29:57.005 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:29:57.005 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:29:57.005 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:29:57.005 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:29:57.005 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:29:57.006 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:29:57.007 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:29:57.007 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:29:57.007 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:29:57.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:29:57.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:29:57.009 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:29:57.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:29:57.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:29:57.010 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:29:57.010 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:29:57.011 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:29:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:29:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T19:29:57.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:29:57.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.013 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46031 2026-02-13T19:29:57.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:29:57.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:29:57.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-13T19:29:57.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:29:57.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.46031 2026-02-13T19:29:57.016 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test2 td/network-ping 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:111: TEST_network_ping_test2: local dir=td/network-ping 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:113: TEST_network_ping_test2: export CEPH_ARGS 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: export 'EXTRA_OPTS= --mon_warn_on_slow_ping_time=0.001' 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: EXTRA_OPTS=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:115: TEST_network_ping_test2: run_mon td/network-ping a 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-13T19:29:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:29:57.036 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --mon_warn_on_slow_ping_time=0.001 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:29:57.059 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:29:57.060 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:29:57.060 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:29:57.060 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:29:57.060 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:29:57.060 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:29:57.060 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:29:57.060 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:29:57.061 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:29:57.061 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:57.061 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.061 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.061 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mon.a.asok 2026-02-13T19:29:57.061 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:29:57.061 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46031/ceph-mon.a.asok config get fsid 2026-02-13T19:29:57.103 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:29:57.103 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:29:57.104 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:29:57.104 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:29:57.104 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46031/ceph-mon.a.asok 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:29:57.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46031/ceph-mon.a.asok config get mon_host 2026-02-13T19:29:57.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:116: TEST_network_ping_test2: run_mgr td/network-ping x 2026-02-13T19:29:57.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-13T19:29:57.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:29:57.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:29:57.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:29:57.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-13T19:29:57.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:29:57.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:29:57.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:29:57.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:29:57.247 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:29:57.247 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.247 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:29:57.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:29:57.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --mon_warn_on_slow_ping_time=0.001 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:29:57.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:117: TEST_network_ping_test2: run_osd td/network-ping 0 2026-02-13T19:29:57.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T19:29:57.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:29:57.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:29:57.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:29:57.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:29:57.267 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:29:57.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-13T19:29:57.269 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:29:57.269 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5887e45f-0ac3-46c7-bc8f-fdb0875c9c4b 2026-02-13T19:29:57.269 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5887e45f-0ac3-46c7-bc8f-fdb0875c9c4b' 2026-02-13T19:29:57.269 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 5887e45f-0ac3-46c7-bc8f-fdb0875c9c4b 2026-02-13T19:29:57.269 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:29:57.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC1e49pc6lyEBAAgUO/cwgFx/rF/DrPWcpVGw== 2026-02-13T19:29:57.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC1e49pc6lyEBAAgUO/cwgFx/rF/DrPWcpVGw=="}' 2026-02-13T19:29:57.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5887e45f-0ac3-46c7-bc8f-fdb0875c9c4b -i td/network-ping/0/new.json 2026-02-13T19:29:57.377 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:29:57.382 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-13T19:29:57.383 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC1e49pc6lyEBAAgUO/cwgFx/rF/DrPWcpVGw== --osd-uuid 5887e45f-0ac3-46c7-bc8f-fdb0875c9c4b 2026-02-13T19:29:57.401 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.399+0000 7f387fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:57.402 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.401+0000 7f387fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:57.403 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.402+0000 7f387fe0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:57.404 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.402+0000 7f387fe0f780 -1 bdev(0x2b38c00 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:29:57.404 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.402+0000 7f387fe0f780 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-13T19:29:57.851 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-13T19:29:57.852 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:29:57.852 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:29:57.853 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:29:57.853 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:29:57.954 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:29:57.955 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:29:57.956 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:29:57.957 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:29:57.957 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:29:57.957 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:29:57.971 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.969+0000 7ff6b9bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:57.973 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.971+0000 7ff6b9bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:57.974 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:57.972+0000 7ff6b9bfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:58.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:29:58.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:29:58.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:29:58.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:29:58.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:29:58.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:29:58.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:29:58.055 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:29:58.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:29:58.056 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:29:58.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:29:58.171 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:58.170+0000 7ff6b9bfd780 -1 Falling back to public interface 2026-02-13T19:29:58.552 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:58.550+0000 7ff6b9bfd780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:29:59.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:29:59.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:29:59.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:29:59.151 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:29:59.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:29:59.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:29:59.251 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2688393303,v1:127.0.0.1:6801/2688393303] [v2:127.0.0.1:6802/2688393303,v1:127.0.0.1:6803/2688393303] exists,up 5887e45f-0ac3-46c7-bc8f-fdb0875c9c4b 2026-02-13T19:29:59.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:29:59.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:29:59.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:29:59.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:118: TEST_network_ping_test2: run_osd td/network-ping 1 2026-02-13T19:29:59.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:29:59.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-13T19:29:59.253 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:29:59.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=957d1359-3469-41f7-a4a5-7a11ff737d21 2026-02-13T19:29:59.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 957d1359-3469-41f7-a4a5-7a11ff737d21' 2026-02-13T19:29:59.254 INFO:tasks.workunit.client.0.trial162.stdout:add osd1 957d1359-3469-41f7-a4a5-7a11ff737d21 2026-02-13T19:29:59.254 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:29:59.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC3e49pDb+5DxAAQ7RBNhrYkk6mgfWtEFE90g== 2026-02-13T19:29:59.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC3e49pDb+5DxAAQ7RBNhrYkk6mgfWtEFE90g=="}' 2026-02-13T19:29:59.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 957d1359-3469-41f7-a4a5-7a11ff737d21 -i td/network-ping/1/new.json 2026-02-13T19:29:59.368 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:29:59.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-13T19:29:59.375 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC3e49pDb+5DxAAQ7RBNhrYkk6mgfWtEFE90g== --osd-uuid 957d1359-3469-41f7-a4a5-7a11ff737d21 2026-02-13T19:29:59.392 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:59.390+0000 7f8d11211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:59.393 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:59.391+0000 7f8d11211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:59.394 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:59.392+0000 7f8d11211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:29:59.394 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:59.393+0000 7f8d11211780 -1 bdev(0x4320c00 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:29:59.395 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:29:59.393+0000 7f8d11211780 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-13T19:29:59.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-13T19:29:59.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:29:59.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:29:59.717 INFO:tasks.workunit.client.0.trial162.stdout:adding osd1 key to auth repository 2026-02-13T19:29:59.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:30:00.040 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:30:00.040 INFO:tasks.workunit.client.0.trial162.stdout:start osd.1 2026-02-13T19:30:00.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:30:00.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:30:00.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:30:00.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:30:00.055 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:00.053+0000 7f5463c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:00.056 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:00.054+0000 7f5463c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:00.057 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:00.055+0000 7f5463c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:00.180 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:00.178+0000 7f5463c2d780 -1 Falling back to public interface 2026-02-13T19:30:00.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:30:00.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:30:00.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:30:00.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:30:00.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:30:00.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:00.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:30:00.267 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:30:00.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:00.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:30:00.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:30:00.499 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:00.498+0000 7f5463c2d780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:30:01.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:30:01.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:01.491 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:30:01.492 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:30:01.492 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:01.492 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:30:01.716 INFO:tasks.workunit.client.0.trial162.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/49099775,v1:127.0.0.1:6811/49099775] [v2:127.0.0.1:6812/49099775,v1:127.0.0.1:6813/49099775] exists,up 957d1359-3469-41f7-a4a5-7a11ff737d21 2026-02-13T19:30:01.719 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:119: TEST_network_ping_test2: run_osd td/network-ping 2 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:30:01.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:30:01.721 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:30:01.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-13T19:30:01.723 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:30:01.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3cd784ee-0c4b-49a3-9bd5-33284119a988 2026-02-13T19:30:01.723 INFO:tasks.workunit.client.0.trial162.stdout:add osd2 3cd784ee-0c4b-49a3-9bd5-33284119a988 2026-02-13T19:30:01.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3cd784ee-0c4b-49a3-9bd5-33284119a988' 2026-02-13T19:30:01.723 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:30:01.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5e49pb5d3KxAAh7+vvv67R/jLmNN0isSzYQ== 2026-02-13T19:30:01.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5e49pb5d3KxAAh7+vvv67R/jLmNN0isSzYQ=="}' 2026-02-13T19:30:01.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3cd784ee-0c4b-49a3-9bd5-33284119a988 -i td/network-ping/2/new.json 2026-02-13T19:30:01.961 INFO:tasks.workunit.client.0.trial162.stdout:2 2026-02-13T19:30:01.968 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-13T19:30:01.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5e49pb5d3KxAAh7+vvv67R/jLmNN0isSzYQ== --osd-uuid 3cd784ee-0c4b-49a3-9bd5-33284119a988 2026-02-13T19:30:01.987 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:01.985+0000 7f5d3f009780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:01.988 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:01.987+0000 7f5d3f009780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:01.989 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:01.988+0000 7f5d3f009780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:01.990 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:01.988+0000 7f5d3f009780 -1 bdev(0x282ac00 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:30:01.990 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:01.988+0000 7f5d3f009780 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-13T19:30:02.310 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-13T19:30:02.310 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:30:02.311 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:30:02.311 INFO:tasks.workunit.client.0.trial162.stdout:adding osd2 key to auth repository 2026-02-13T19:30:02.311 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:30:02.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:30:02.634 INFO:tasks.workunit.client.0.trial162.stdout:start osd.2 2026-02-13T19:30:02.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=3f7d371a-280c-44e6-9ea0-0affa64ca54a --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.46031/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:30:02.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:30:02.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:30:02.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:30:02.649 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:02.648+0000 7f2526e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:02.650 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:02.649+0000 7f2526e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:02.651 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:02.650+0000 7f2526e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:02.787 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:02.785+0000 7f2526e0f780 -1 Falling back to public interface 2026-02-13T19:30:02.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:30:02.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:30:02.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:30:02.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:30:02.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:30:02.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:02.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:30:02.863 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:30:02.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:02.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:30:03.091 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:03.089+0000 7f2526e0f780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:30:03.091 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:30:04.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:30:04.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:04.093 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:30:04.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:30:04.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:04.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:30:04.319 INFO:tasks.workunit.client.0.trial162.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2098373611,v1:127.0.0.1:6819/2098373611] [v2:127.0.0.1:6820/2098373611,v1:127.0.0.1:6821/2098373611] exists,up 3cd784ee-0c4b-49a3-9bd5-33284119a988 2026-02-13T19:30:04.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:30:04.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:30:04.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:30:04.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:121: TEST_network_ping_test2: sleep 5 2026-02-13T19:30:09.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:122: TEST_network_ping_test2: ceph osd crush add-bucket dc1 datacenter 2026-02-13T19:30:09.593 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'dc1' already exists 2026-02-13T19:30:09.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:123: TEST_network_ping_test2: ceph osd crush add-bucket dc2 datacenter 2026-02-13T19:30:09.876 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'dc2' already exists 2026-02-13T19:30:09.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:124: TEST_network_ping_test2: ceph osd crush add-bucket dc3 datacenter 2026-02-13T19:30:10.161 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'dc3' already exists 2026-02-13T19:30:10.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:125: TEST_network_ping_test2: ceph osd crush add-bucket rack1 rack 2026-02-13T19:30:10.444 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'rack1' already exists 2026-02-13T19:30:10.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:126: TEST_network_ping_test2: ceph osd crush add-bucket rack2 rack 2026-02-13T19:30:10.727 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'rack2' already exists 2026-02-13T19:30:10.734 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:127: TEST_network_ping_test2: ceph osd crush add-bucket rack3 rack 2026-02-13T19:30:11.007 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'rack3' already exists 2026-02-13T19:30:11.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:128: TEST_network_ping_test2: ceph osd crush add-bucket host1 host 2026-02-13T19:30:11.291 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'host1' already exists 2026-02-13T19:30:11.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:129: TEST_network_ping_test2: ceph osd crush add-bucket host2 host 2026-02-13T19:30:11.570 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'host2' already exists 2026-02-13T19:30:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:130: TEST_network_ping_test2: ceph osd crush add-bucket host3 host 2026-02-13T19:30:11.852 INFO:tasks.workunit.client.0.trial162.stderr:bucket 'host3' already exists 2026-02-13T19:30:11.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:131: TEST_network_ping_test2: ceph osd crush move dc1 root=default 2026-02-13T19:30:12.135 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -5 name 'dc1' to location {root=default} in crush map 2026-02-13T19:30:12.143 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:132: TEST_network_ping_test2: ceph osd crush move dc2 root=default 2026-02-13T19:30:12.423 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -6 name 'dc2' to location {root=default} in crush map 2026-02-13T19:30:12.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:133: TEST_network_ping_test2: ceph osd crush move dc3 root=default 2026-02-13T19:30:12.708 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -7 name 'dc3' to location {root=default} in crush map 2026-02-13T19:30:12.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:134: TEST_network_ping_test2: ceph osd crush move rack1 datacenter=dc1 2026-02-13T19:30:12.993 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -8 name 'rack1' to location {datacenter=dc1} in crush map 2026-02-13T19:30:13.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:135: TEST_network_ping_test2: ceph osd crush move rack2 datacenter=dc2 2026-02-13T19:30:13.282 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -9 name 'rack2' to location {datacenter=dc2} in crush map 2026-02-13T19:30:13.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:136: TEST_network_ping_test2: ceph osd crush move rack3 datacenter=dc3 2026-02-13T19:30:13.586 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -10 name 'rack3' to location {datacenter=dc3} in crush map 2026-02-13T19:30:13.594 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:137: TEST_network_ping_test2: ceph osd crush move host1 rack=rack1 2026-02-13T19:30:13.868 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -11 name 'host1' to location {rack=rack1} in crush map 2026-02-13T19:30:13.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:138: TEST_network_ping_test2: ceph osd crush move host2 rack=rack2 2026-02-13T19:30:14.150 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -12 name 'host2' to location {rack=rack2} in crush map 2026-02-13T19:30:14.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:139: TEST_network_ping_test2: ceph osd crush move host3 rack=rack3 2026-02-13T19:30:14.435 INFO:tasks.workunit.client.0.trial162.stderr:no need to move item id -13 name 'host3' to location {rack=rack3} in crush map 2026-02-13T19:30:14.442 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:140: TEST_network_ping_test2: ceph osd crush set osd.0 1.0 host=host1 2026-02-13T19:30:14.718 INFO:tasks.workunit.client.0.trial162.stderr:set item id 0 name 'osd.0' weight 1 at location {host=host1}: no change 2026-02-13T19:30:14.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:141: TEST_network_ping_test2: ceph osd crush set osd.1 1.0 host=host2 2026-02-13T19:30:15.000 INFO:tasks.workunit.client.0.trial162.stderr:set item id 1 name 'osd.1' weight 1 at location {host=host2}: no change 2026-02-13T19:30:15.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:142: TEST_network_ping_test2: ceph osd crush set osd.2 1.0 host=host3 2026-02-13T19:30:15.282 INFO:tasks.workunit.client.0.trial162.stderr:set item id 2 name 'osd.2' weight 1 at location {host=host3}: no change 2026-02-13T19:30:15.290 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:143: TEST_network_ping_test2: ceph osd crush rule create-simple myrule default host firstn 2026-02-13T19:30:15.566 INFO:tasks.workunit.client.0.trial162.stderr:rule myrule already exists 2026-02-13T19:30:15.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:145: TEST_network_ping_test2: create_pool foo 16 16 replicated myrule 2026-02-13T19:30:15.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 16 replicated myrule 2026-02-13T19:30:15.853 INFO:tasks.workunit.client.0.trial162.stderr:pool 'foo' already exists 2026-02-13T19:30:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:30:16.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:148: TEST_network_ping_test2: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-13T19:30:16.885 INFO:tasks.workunit.client.0.trial162.stdout:hints = 1 2026-02-13T19:30:16.886 INFO:tasks.workunit.client.0.trial162.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-13T19:30:16.886 INFO:tasks.workunit.client.0.trial162.stdout:Object prefix: benchmark_data_trial162_56580 2026-02-13T19:30:16.886 INFO:tasks.workunit.client.0.trial162.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-13T19:30:16.886 INFO:tasks.workunit.client.0.trial162.stdout: 0 0 0 0 0 0 - 0 2026-02-13T19:30:17.885 INFO:tasks.workunit.client.0.trial162.stdout: 1 16 11589 11573 45.2034 45.207 0.000824053 0.0013805 2026-02-13T19:30:18.885 INFO:tasks.workunit.client.0.trial162.stdout: 2 16 23790 23774 46.43 47.6602 0.00203401 0.00134435 2026-02-13T19:30:19.885 INFO:tasks.workunit.client.0.trial162.stdout: 3 16 35908 35892 46.7307 47.3359 0.00142788 0.00133461 2026-02-13T19:30:20.885 INFO:tasks.workunit.client.0.trial162.stdout: 4 16 47662 47646 46.5257 45.9141 0.000833841 0.00134192 2026-02-13T19:30:21.885 INFO:tasks.workunit.client.0.trial162.stdout: 5 16 59700 59684 46.6245 47.0234 0.000832418 0.00133907 2026-02-13T19:30:22.885 INFO:tasks.workunit.client.0.trial162.stdout: 6 16 72102 72086 46.9273 48.4453 0.00151644 0.00133037 2026-02-13T19:30:23.885 INFO:tasks.workunit.client.0.trial162.stdout: 7 15 84075 84060 46.9048 46.7734 0.00169431 0.00133115 2026-02-13T19:30:24.885 INFO:tasks.workunit.client.0.trial162.stdout: 8 16 94675 94659 46.2166 41.4023 0.00113614 0.00135098 2026-02-13T19:30:25.885 INFO:tasks.workunit.client.0.trial162.stdout: 9 16 105310 105294 45.6969 41.543 0.00109427 0.00136634 2026-02-13T19:30:26.885 INFO:tasks.workunit.client.0.trial162.stdout: 10 3 115839 115836 45.2449 41.1797 0.00113002 0.00138004 2026-02-13T19:30:27.885 INFO:tasks.workunit.client.0.trial162.stdout:Total time run: 10.0009 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Total writes made: 115839 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Write size: 4096 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Object size: 4096 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Bandwidth (MB/sec): 45.2456 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Stddev Bandwidth: 2.81647 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Max bandwidth (MB/sec): 48.4453 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Min bandwidth (MB/sec): 41.1797 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Average IOPS: 11582 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Stddev IOPS: 721.017 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Max IOPS: 12402 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Min IOPS: 10542 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Average Latency(s): 0.00138003 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Stddev Latency(s): 0.000664922 2026-02-13T19:30:27.886 INFO:tasks.workunit.client.0.trial162.stdout:Max latency(s): 0.0119674 2026-02-13T19:30:27.887 INFO:tasks.workunit.client.0.trial162.stdout:Min latency(s): 0.000445408 2026-02-13T19:30:27.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:151: TEST_network_ping_test2: sleep 10 2026-02-13T19:30:37.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:152: TEST_network_ping_test2: flush_pg_stats 2026-02-13T19:30:37.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:30:37.891 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:30:38.117 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:30:38.117 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:30:38.117 INFO:tasks.workunit.client.0.trial162.stderr:2' 2026-02-13T19:30:38.117 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:30:38.117 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:30:38.117 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:30:38.184 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-13T19:30:38.185 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-13T19:30:38.185 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-13T19:30:38.185 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:30:38.185 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:30:38.250 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-13T19:30:38.250 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-13T19:30:38.250 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672970' 2026-02-13T19:30:38.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:30:38.251 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:30:38.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-13T19:30:38.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-13T19:30:38.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672970 2-60129542153' 2026-02-13T19:30:38.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:30:38.315 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-13T19:30:38.315 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:30:38.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:30:38.316 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-13T19:30:38.317 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:30:38.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-13T19:30:38.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-13T19:30:38.318 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836490 2026-02-13T19:30:38.318 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:30:38.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T19:30:38.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:30:39.553 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:30:39.553 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:30:39.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-13T19:30:39.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:30:39.785 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672970 2026-02-13T19:30:39.785 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:30:39.786 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:30:39.786 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-13T19:30:39.786 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:30:39.787 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672970 2026-02-13T19:30:39.788 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-13T19:30:39.788 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 42949672970 2026-02-13T19:30:39.788 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:30:40.016 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-13T19:30:40.016 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:30:40.017 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542153 2026-02-13T19:30:40.017 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:30:40.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:30:40.018 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-13T19:30:40.018 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:30:40.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542153 2026-02-13T19:30:40.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-13T19:30:40.019 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542153 2026-02-13T19:30:40.020 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:30:40.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-13T19:30:40.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: ceph health 2026-02-13T19:30:40.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: tee td/network-ping/health 2026-02-13T19:30:40.532 INFO:tasks.workunit.client.0.trial162.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.324ms); Slow OSD heartbeats on front (longest 0.344ms) 2026-02-13T19:30:40.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:155: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats' td/network-ping/health 2026-02-13T19:30:40.542 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: ceph health detail 2026-02-13T19:30:40.542 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: tee td/network-ping/health 2026-02-13T19:30:40.854 INFO:tasks.workunit.client.0.trial162.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.324ms); Slow OSD heartbeats on front (longest 0.344ms) 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout:[WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 0.324ms) 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.324 msec possibly improving 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.225 msec 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.210 msec 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.200 msec 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.160 msec 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.116 msec 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout:[WRN] OSD_SLOW_PING_TIME_FRONT: Slow OSD heartbeats on front (longest 0.344ms) 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.344 msec possibly improving 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.216 msec 2026-02-13T19:30:40.855 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.215 msec 2026-02-13T19:30:40.856 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.210 msec 2026-02-13T19:30:40.856 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.158 msec 2026-02-13T19:30:40.856 INFO:tasks.workunit.client.0.trial162.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.152 msec 2026-02-13T19:30:40.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:158: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_BACK td/network-ping/health 2026-02-13T19:30:40.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:159: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_FRONT td/network-ping/health 2026-02-13T19:30:40.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:160: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats on front from osd[.][0-2] [[]dc[1-3],rack[1-3][]] to osd[.][0-2] [[]dc[1-3],rack[1-3][]]' td/network-ping/health 2026-02-13T19:30:40.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:162: TEST_network_ping_test2: rm -f td/network-ping/health 2026-02-13T19:30:40.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-13T19:30:40.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T19:30:40.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:30:40.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T19:30:40.867 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:30:40.867 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:30:40.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:30:40.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:30:40.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:30:40.974 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:30:40.974 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:30:40.974 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:30:40.975 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:30:40.976 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:30:40.976 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:30:40.976 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:30:40.977 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:40.977 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:30:40.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:30:40.978 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:40.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:30:40.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:30:40.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T19:30:41.708 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:30:41.708 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:41.708 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:30:41.708 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46031 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/network-ping 0 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-13T19:30:41.710 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:30:41.711 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:30:41.711 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:30:41.711 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:30:41.711 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:30:41.711 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:30:41.712 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:30:41.712 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:30:41.712 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:30:41.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:30:41.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:30:41.714 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:30:41.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:41.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:30:41.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:30:41.715 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:41.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:30:41.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T19:30:41.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-13T19:30:41.717 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:30:41.718 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:41.718 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46031 2026-02-13T19:30:41.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46031 2026-02-13T19:30:41.719 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:30:41.719 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:30:41.719 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-13T19:30:41.720 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T19:30:41.720 DEBUG:teuthology.orchestra.run.trial162:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T19:30:41.786 INFO:tasks.workunit:Running workunit misc/ok-to-stop.sh... 2026-02-13T19:30:41.786 DEBUG:teuthology.orchestra.run.trial162:workunit test misc/ok-to-stop.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh 2026-02-13T19:30:41.849 INFO:tasks.workunit.client.0.trial162.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:30:41.858 INFO:tasks.workunit.client.0.trial162.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:30:41.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:30:41.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:30:41.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-13T19:30:41.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-13T19:30:41.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-13T19:30:41.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-13T19:30:41.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-13T19:30:41.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-13T19:30:41.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/ok-to-stop 2026-02-13T19:30:41.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:6: run: local dir=td/ok-to-stop 2026-02-13T19:30:41.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:7: run: shift 2026-02-13T19:30:41.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: export CEPH_MON_A=127.0.0.1:7150 2026-02-13T19:30:41.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: CEPH_MON_A=127.0.0.1:7150 2026-02-13T19:30:41.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: export CEPH_MON_B=127.0.0.1:7151 2026-02-13T19:30:41.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: CEPH_MON_B=127.0.0.1:7151 2026-02-13T19:30:41.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: export CEPH_MON_C=127.0.0.1:7152 2026-02-13T19:30:41.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: CEPH_MON_C=127.0.0.1:7152 2026-02-13T19:30:41.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: export CEPH_MON_D=127.0.0.1:7153 2026-02-13T19:30:41.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: CEPH_MON_D=127.0.0.1:7153 2026-02-13T19:30:41.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: export CEPH_MON_E=127.0.0.1:7154 2026-02-13T19:30:41.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: CEPH_MON_E=127.0.0.1:7154 2026-02-13T19:30:41.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:14: run: export CEPH_ARGS 2026-02-13T19:30:41.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: uuidgen 2026-02-13T19:30:41.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: CEPH_ARGS+='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none ' 2026-02-13T19:30:41.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: export 'ORIG_CEPH_ARGS=--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none ' 2026-02-13T19:30:41.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: ORIG_CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none ' 2026-02-13T19:30:41.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: set 2026-02-13T19:30:41.864 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T19:30:41.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: local 'funcs=TEST_0_mds 2026-02-13T19:30:41.864 INFO:tasks.workunit.client.0.trial162.stderr:TEST_0_osd 2026-02-13T19:30:41.864 INFO:tasks.workunit.client.0.trial162.stderr:TEST_1_mon_checks 2026-02-13T19:30:41.864 INFO:tasks.workunit.client.0.trial162.stderr:TEST_2_mons_checks 2026-02-13T19:30:41.865 INFO:tasks.workunit.client.0.trial162.stderr:TEST_3_mons_checks 2026-02-13T19:30:41.865 INFO:tasks.workunit.client.0.trial162.stderr:TEST_4_mons_checks 2026-02-13T19:30:41.865 INFO:tasks.workunit.client.0.trial162.stderr:TEST_5_mons_checks' 2026-02-13T19:30:41.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T19:30:41.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T19:30:41.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T19:30:41.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T19:30:41.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:30:41.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:30:41.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:30:41.866 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:30:41.866 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:30:41.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:30:41.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:30:41.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:30:41.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:30:41.867 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:30:41.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:30:41.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:30:41.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:30:41.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:30:41.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:30:41.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:41.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:30:41.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:30:41.877 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:41.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:30:41.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:30:41.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:30:41.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:30:41.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:41.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:41.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:30:41.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:30:41.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:30:41.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T19:30:41.879 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:30:41.879 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:41.879 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:41.879 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56967 2026-02-13T19:30:41.880 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:30:41.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:30:41.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:30:41.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:30:41.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T19:30:41.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_mds td/ok-to-stop 2026-02-13T19:30:41.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:208: TEST_0_mds: local dir=td/ok-to-stop 2026-02-13T19:30:41.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:210: TEST_0_mds: CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:30:41.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:212: TEST_0_mds: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T19:30:41.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:30:41.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:30:41.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:30:41.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:30:41.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T19:30:41.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T19:30:41.887 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:30:41.887 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:41.887 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:41.887 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:41.888 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:41.888 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:41.888 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:41.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T19:30:41.908 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:30:41.910 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:41.911 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:41.912 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:30:41.912 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:30:41.912 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get fsid 2026-02-13T19:30:41.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:30:41.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:30:41.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:30:41.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:30:41.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:30:41.954 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get mon_host 2026-02-13T19:30:41.996 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:213: TEST_0_mds: run_mgr td/ok-to-stop x 2026-02-13T19:30:41.996 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-13T19:30:41.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:30:41.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:30:41.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:30:41.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-13T19:30:41.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:30:42.091 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:30:42.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:42.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:42.092 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:42.092 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:42.092 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:42.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:42.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:30:42.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:30:42.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:214: TEST_0_mds: run_osd td/ok-to-stop 0 2026-02-13T19:30:42.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T19:30:42.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:30:42.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:30:42.111 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:30:42.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:30:42.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a7fbee6d-7f97-4e24-9821-d354072cc9b2 2026-02-13T19:30:42.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a7fbee6d-7f97-4e24-9821-d354072cc9b2' 2026-02-13T19:30:42.115 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 a7fbee6d-7f97-4e24-9821-d354072cc9b2 2026-02-13T19:30:42.115 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:30:42.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDie49pRVIZBxAAcxDb5Ng27VuPZgWCQ0T/ow== 2026-02-13T19:30:42.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDie49pRVIZBxAAcxDb5Ng27VuPZgWCQ0T/ow=="}' 2026-02-13T19:30:42.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a7fbee6d-7f97-4e24-9821-d354072cc9b2 -i td/ok-to-stop/0/new.json 2026-02-13T19:30:42.221 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:30:42.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-13T19:30:42.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDie49pRVIZBxAAcxDb5Ng27VuPZgWCQ0T/ow== --osd-uuid a7fbee6d-7f97-4e24-9821-d354072cc9b2 2026-02-13T19:30:42.244 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.242+0000 7f825783b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:42.245 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.244+0000 7f825783b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:42.246 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.245+0000 7f825783b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:42.246 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.245+0000 7f825783b780 -1 bdev(0x3de4c00 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:30:42.246 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.245+0000 7f825783b780 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-13T19:30:42.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-13T19:30:42.655 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:30:42.655 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:30:42.655 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:30:42.656 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:30:42.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:30:42.760 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:30:42.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:30:42.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:30:42.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:30:42.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:30:42.774 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.772+0000 7ffa0fa47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:42.775 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.774+0000 7ffa0fa47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:42.776 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.774+0000 7ffa0fa47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:42.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:30:42.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:30:42.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:30:42.857 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:30:42.857 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:30:42.857 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:42.857 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:30:42.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:30:42.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:42.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:30:42.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:30:42.983 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:42.982+0000 7ffa0fa47780 -1 Falling back to public interface 2026-02-13T19:30:43.338 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:43.336+0000 7ffa0fa47780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:30:43.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:30:43.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:43.954 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:30:43.954 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:30:43.954 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:43.955 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:30:44.049 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2016476662,v1:127.0.0.1:6801/2016476662] [v2:127.0.0.1:6802/2016476662,v1:127.0.0.1:6803/2016476662] exists,up a7fbee6d-7f97-4e24-9821-d354072cc9b2 2026-02-13T19:30:44.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:30:44.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:30:44.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:30:44.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:215: TEST_0_mds: run_mds td/ok-to-stop a 2026-02-13T19:30:44.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-13T19:30:44.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=a 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/a 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:44.052 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id a --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-13T19:30:44.064 INFO:tasks.workunit.client.0.trial162.stdout:starting mds.a at 2026-02-13T19:30:44.066 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:217: TEST_0_mds: ceph osd pool create meta 1 2026-02-13T19:30:44.255 INFO:tasks.workunit.client.0.trial162.stderr:pool 'meta' already exists 2026-02-13T19:30:44.261 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:218: TEST_0_mds: ceph osd pool create data 1 2026-02-13T19:30:44.503 INFO:tasks.workunit.client.0.trial162.stderr:pool 'data' already exists 2026-02-13T19:30:44.510 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:219: TEST_0_mds: ceph fs new myfs meta data 2026-02-13T19:30:44.848 INFO:tasks.workunit.client.0.trial162.stderr:filesystem 'myfs' already exists 2026-02-13T19:30:44.855 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:220: TEST_0_mds: sleep 5 2026-02-13T19:30:49.857 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:222: TEST_0_mds: ceph mds ok-to-stop a 2026-02-13T19:30:50.059 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 2026-02-13T19:30:50.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:223: TEST_0_mds: ceph mds ok-to-stop a dne 2026-02-13T19:30:50.263 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 2026-02-13T19:30:50.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:224: TEST_0_mds: ceph mds ok-to-stop dne 2026-02-13T19:30:50.581 INFO:tasks.workunit.client.0.trial162.stderr:should be safe to stop [dne] 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:226: TEST_0_mds: run_mds td/ok-to-stop b 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=b 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/b 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:50.590 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:50.591 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:50.591 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:50.591 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:50.591 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:50.591 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id b --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-13T19:30:50.604 INFO:tasks.workunit.client.0.trial162.stdout:starting mds.b at 2026-02-13T19:30:50.606 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:227: TEST_0_mds: sleep 5 2026-02-13T19:30:55.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:229: TEST_0_mds: ceph mds ok-to-stop a 2026-02-13T19:30:55.919 INFO:tasks.workunit.client.0.trial162.stderr:should be safe to stop [a] 2026-02-13T19:30:55.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:230: TEST_0_mds: ceph mds ok-to-stop b 2026-02-13T19:30:56.240 INFO:tasks.workunit.client.0.trial162.stderr:should be safe to stop [b] 2026-02-13T19:30:56.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:231: TEST_0_mds: ceph mds ok-to-stop a b 2026-02-13T19:30:56.450 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 4135 2026-02-13T19:30:56.454 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:232: TEST_0_mds: ceph mds ok-to-stop a dne1 dne2 2026-02-13T19:30:56.767 INFO:tasks.workunit.client.0.trial162.stderr:should be safe to stop [a,dne1,dne2] 2026-02-13T19:30:56.774 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:233: TEST_0_mds: ceph mds ok-to-stop b dne 2026-02-13T19:30:57.094 INFO:tasks.workunit.client.0.trial162.stderr:should be safe to stop [b,dne] 2026-02-13T19:30:57.101 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:234: TEST_0_mds: ceph mds ok-to-stop a b dne 2026-02-13T19:30:57.306 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4115 and/or standby gids 4135 2026-02-13T19:30:57.309 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:235: TEST_0_mds: ceph mds ok-to-stop dne1 dne2 2026-02-13T19:30:57.625 INFO:tasks.workunit.client.0.trial162.stderr:should be safe to stop [dne1,dne2] 2026-02-13T19:30:57.632 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:237: TEST_0_mds: kill_daemons td/ok-to-stop KILL mds.a 2026-02-13T19:30:57.633 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:30:57.634 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:30:57.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:30:57.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:30:57.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:30:57.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:30:57.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T19:30:57.739 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:30:57.739 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:30:57.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:30:57.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:30:57.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:30:57.848 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:30:57.848 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T19:30:57.848 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:30:57.848 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:30:57.848 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:30:57.849 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:30:57.849 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:30:57.849 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:30:57.849 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:30:57.849 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:30:57.852 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:30:57.853 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:30:57.853 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:30:57.854 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:30:57.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:30:57.855 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:30:57.855 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:30:57.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:57.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:30:57.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:30:57.856 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:57.857 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:30:57.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:30:57.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:30:57.862 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:30:57.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:57.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:57.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:30:57.865 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:30:57.866 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:30:57.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:30:57.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:30:57.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:30:57.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:30:57.867 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:30:57.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:30:57.867 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:30:57.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:30:57.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:30:57.868 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:30:57.869 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:57.869 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:30:57.869 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:30:57.869 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:30:57.870 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:30:57.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:30:57.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:30:57.872 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:30:57.872 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:57.872 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:57.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:30:57.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:30:57.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:30:57.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T19:30:57.874 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:30:57.874 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:57.874 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:57.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56967 2026-02-13T19:30:57.875 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_osd td/ok-to-stop 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:241: TEST_0_osd: local dir=td/ok-to-stop 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:243: TEST_0_osd: CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:245: TEST_0_osd: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:30:57.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:30:57.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:30:57.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:30:57.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T19:30:57.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T19:30:57.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:30:57.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:57.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:57.894 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:57.894 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:57.894 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:57.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:57.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T19:30:57.918 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:30:57.919 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:30:57.920 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:30:57.920 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:57.920 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:57.920 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:30:57.920 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:30:57.920 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get fsid 2026-02-13T19:30:57.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:30:57.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:30:57.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:30:57.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:30:57.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:30:57.964 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:30:57.964 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:30:57.964 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:30:57.964 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:30:57.964 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:57.964 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:57.965 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:30:57.965 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:30:57.965 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get mon_host 2026-02-13T19:30:58.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:246: TEST_0_osd: run_mgr td/ok-to-stop x 2026-02-13T19:30:58.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-13T19:30:58.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:30:58.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:30:58.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:30:58.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-13T19:30:58.010 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:30:58.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:30:58.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:58.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:58.106 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:58.106 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:58.106 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:58.106 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:58.106 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:30:58.107 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:30:58.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:247: TEST_0_osd: run_osd td/ok-to-stop 0 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:30:58.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:30:58.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:30:58.125 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-13T19:30:58.125 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:30:58.125 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4d9e805e-7309-4d98-a9bb-2738aceca862 2026-02-13T19:30:58.125 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4d9e805e-7309-4d98-a9bb-2738aceca862' 2026-02-13T19:30:58.125 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 4d9e805e-7309-4d98-a9bb-2738aceca862 2026-02-13T19:30:58.125 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:30:58.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDye49pchTxBxAAKKOa1rzRCx5BbgfTSVnYOQ== 2026-02-13T19:30:58.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDye49pchTxBxAAKKOa1rzRCx5BbgfTSVnYOQ=="}' 2026-02-13T19:30:58.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4d9e805e-7309-4d98-a9bb-2738aceca862 -i td/ok-to-stop/0/new.json 2026-02-13T19:30:58.235 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:30:58.241 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-13T19:30:58.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDye49pchTxBxAAKKOa1rzRCx5BbgfTSVnYOQ== --osd-uuid 4d9e805e-7309-4d98-a9bb-2738aceca862 2026-02-13T19:30:58.257 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.256+0000 7f5f0c80b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:58.259 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.257+0000 7f5f0c80b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:58.259 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.258+0000 7f5f0c80b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:58.260 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.259+0000 7f5f0c80b780 -1 bdev(0x3bdec00 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:30:58.260 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.259+0000 7f5f0c80b780 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-13T19:30:58.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-13T19:30:58.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:30:58.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:30:58.693 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:30:58.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:30:58.793 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:30:58.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:30:58.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:30:58.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:30:58.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:30:58.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:30:58.807 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.806+0000 7f3684663780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:58.809 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.807+0000 7f3684663780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:58.809 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:58.808+0000 7f3684663780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:30:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:30:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:30:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:30:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:30:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:30:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:30:58.891 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:30:58.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:58.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:30:58.988 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:30:59.011 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:59.010+0000 7f3684663780 -1 Falling back to public interface 2026-02-13T19:30:59.357 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:30:59.355+0000 7f3684663780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:30:59.989 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:30:59.989 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:30:59.989 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:30:59.989 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:30:59.989 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:30:59.989 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:31:00.086 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/179116137,v1:127.0.0.1:6801/179116137] [v2:127.0.0.1:6802/179116137,v1:127.0.0.1:6803/179116137] exists,up 4d9e805e-7309-4d98-a9bb-2738aceca862 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:248: TEST_0_osd: run_osd td/ok-to-stop 1 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/1 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:31:00.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/1' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/1/journal' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:00.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/1 2026-02-13T19:31:00.089 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:31:00.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=23ea3f9c-c554-45fc-a1a3-4709b737e7cf 2026-02-13T19:31:00.090 INFO:tasks.workunit.client.0.trial162.stdout:add osd1 23ea3f9c-c554-45fc-a1a3-4709b737e7cf 2026-02-13T19:31:00.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 23ea3f9c-c554-45fc-a1a3-4709b737e7cf' 2026-02-13T19:31:00.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:31:00.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD0e49pLUvTBRAAf7s9z+P1jYw66sv1AFC49A== 2026-02-13T19:31:00.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD0e49pLUvTBRAAf7s9z+P1jYw66sv1AFC49A=="}' 2026-02-13T19:31:00.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 23ea3f9c-c554-45fc-a1a3-4709b737e7cf -i td/ok-to-stop/1/new.json 2026-02-13T19:31:00.200 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:31:00.206 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/1/new.json 2026-02-13T19:31:00.207 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD0e49pLUvTBRAAf7s9z+P1jYw66sv1AFC49A== --osd-uuid 23ea3f9c-c554-45fc-a1a3-4709b737e7cf 2026-02-13T19:31:00.221 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.220+0000 7f967ec11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:00.224 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.222+0000 7f967ec11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:00.225 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.223+0000 7f967ec11780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:00.225 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.224+0000 7f967ec11780 -1 bdev(0x437ac00 td/ok-to-stop/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:31:00.225 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.224+0000 7f967ec11780 -1 bluestore(td/ok-to-stop/1) _read_fsid unparsable uuid 2026-02-13T19:31:00.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/1/keyring 2026-02-13T19:31:00.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:31:00.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:31:00.567 INFO:tasks.workunit.client.0.trial162.stdout:adding osd1 key to auth repository 2026-02-13T19:31:00.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:31:00.884 INFO:tasks.workunit.client.0.trial162.stdout:start osd.1 2026-02-13T19:31:00.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:31:00.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:31:00.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:31:00.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:31:00.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:31:00.898 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.896+0000 7ff4345d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:00.899 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.898+0000 7ff4345d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:00.900 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:00.898+0000 7ff4345d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:01.054 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:01.052+0000 7ff4345d6780 -1 Falling back to public interface 2026-02-13T19:31:01.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:31:01.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:31:01.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:31:01.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:31:01.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:31:01.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:31:01.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:31:01.117 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:31:01.117 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:31:01.117 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:31:01.342 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:01.340+0000 7ff4345d6780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:31:01.347 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:31:02.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:31:02.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:31:02.349 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:31:02.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:31:02.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:31:02.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:31:02.582 INFO:tasks.workunit.client.0.trial162.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1263645909,v1:127.0.0.1:6811/1263645909] [v2:127.0.0.1:6812/1263645909,v1:127.0.0.1:6813/1263645909] exists,up 23ea3f9c-c554-45fc-a1a3-4709b737e7cf 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:249: TEST_0_osd: run_osd td/ok-to-stop 2 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/2 2026-02-13T19:31:02.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/2' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/2/journal' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:31:02.587 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T19:31:02.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/2 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ccab5d3b-245a-4ec1-8d39-b25c590a93ef 2026-02-13T19:31:02.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 ccab5d3b-245a-4ec1-8d39-b25c590a93ef' 2026-02-13T19:31:02.590 INFO:tasks.workunit.client.0.trial162.stdout:add osd2 ccab5d3b-245a-4ec1-8d39-b25c590a93ef 2026-02-13T19:31:02.590 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:31:02.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD2e49pbLZnIxAAM7DeT+TmG6cj7ZTZgV+OMw== 2026-02-13T19:31:02.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD2e49pbLZnIxAAM7DeT+TmG6cj7ZTZgV+OMw=="}' 2026-02-13T19:31:02.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ccab5d3b-245a-4ec1-8d39-b25c590a93ef -i td/ok-to-stop/2/new.json 2026-02-13T19:31:02.829 INFO:tasks.workunit.client.0.trial162.stdout:2 2026-02-13T19:31:02.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/2/new.json 2026-02-13T19:31:02.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD2e49pbLZnIxAAM7DeT+TmG6cj7ZTZgV+OMw== --osd-uuid ccab5d3b-245a-4ec1-8d39-b25c590a93ef 2026-02-13T19:31:02.853 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:02.851+0000 7fd70b437780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:02.854 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:02.853+0000 7fd70b437780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:02.855 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:02.853+0000 7fd70b437780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:02.855 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:02.854+0000 7fd70b437780 -1 bdev(0x3702c00 td/ok-to-stop/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:31:02.856 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:02.854+0000 7fd70b437780 -1 bluestore(td/ok-to-stop/2) _read_fsid unparsable uuid 2026-02-13T19:31:03.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/2/keyring 2026-02-13T19:31:03.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:31:03.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:31:03.183 INFO:tasks.workunit.client.0.trial162.stdout:adding osd2 key to auth repository 2026-02-13T19:31:03.184 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:31:03.505 INFO:tasks.workunit.client.0.trial162.stdout:start osd.2 2026-02-13T19:31:03.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:31:03.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:31:03.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:31:03.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:31:03.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:31:03.519 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:03.517+0000 7f7f41c4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:03.520 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:03.519+0000 7f7f41c4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:03.521 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:03.520+0000 7f7f41c4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:03.687 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:03.686+0000 7f7f41c4a780 -1 Falling back to public interface 2026-02-13T19:31:03.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:31:03.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:31:03.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:31:03.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:31:03.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:31:03.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:31:03.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:31:03.733 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:31:03.733 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:31:03.733 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:31:03.966 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:31:04.052 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:04.050+0000 7f7f41c4a780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:31:04.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:31:04.968 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:31:04.968 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:31:04.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:31:04.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:31:04.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:31:05.198 INFO:tasks.workunit.client.0.trial162.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/447263005,v1:127.0.0.1:6819/447263005] [v2:127.0.0.1:6820/447263005,v1:127.0.0.1:6821/447263005] exists,up ccab5d3b-245a-4ec1-8d39-b25c590a93ef 2026-02-13T19:31:05.202 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:250: TEST_0_osd: run_osd td/ok-to-stop 3 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/3 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:31:05.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/3' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/3/journal' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:05.204 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:31:05.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:31:05.206 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/3 2026-02-13T19:31:05.206 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:31:05.206 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5a49bb18-a42d-4fa8-a99c-537a3018a808 2026-02-13T19:31:05.206 INFO:tasks.workunit.client.0.trial162.stdout:add osd3 5a49bb18-a42d-4fa8-a99c-537a3018a808 2026-02-13T19:31:05.206 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 5a49bb18-a42d-4fa8-a99c-537a3018a808' 2026-02-13T19:31:05.207 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:31:05.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD5e49pBrOUDBAA0k9D6oN69NzsXsLjahrdWA== 2026-02-13T19:31:05.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD5e49pBrOUDBAA0k9D6oN69NzsXsLjahrdWA=="}' 2026-02-13T19:31:05.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5a49bb18-a42d-4fa8-a99c-537a3018a808 -i td/ok-to-stop/3/new.json 2026-02-13T19:31:05.446 INFO:tasks.workunit.client.0.trial162.stdout:3 2026-02-13T19:31:05.453 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/3/new.json 2026-02-13T19:31:05.454 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5e49pBrOUDBAA0k9D6oN69NzsXsLjahrdWA== --osd-uuid 5a49bb18-a42d-4fa8-a99c-537a3018a808 2026-02-13T19:31:05.469 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:05.468+0000 7f79dd039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:05.470 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:05.469+0000 7f79dd039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:05.471 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:05.470+0000 7f79dd039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:05.472 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:05.470+0000 7f79dd039780 -1 bdev(0x266ac00 td/ok-to-stop/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:31:05.472 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:05.470+0000 7f79dd039780 -1 bluestore(td/ok-to-stop/3) _read_fsid unparsable uuid 2026-02-13T19:31:05.803 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/3/keyring 2026-02-13T19:31:05.803 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:31:05.804 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:31:05.804 INFO:tasks.workunit.client.0.trial162.stdout:adding osd3 key to auth repository 2026-02-13T19:31:05.804 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:31:06.123 INFO:tasks.workunit.client.0.trial162.stdout:start osd.3 2026-02-13T19:31:06.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:31:06.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:31:06.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:31:06.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:31:06.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:31:06.137 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:06.135+0000 7f60f3222780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:06.138 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:06.137+0000 7f60f3222780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:06.139 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:06.138+0000 7f60f3222780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:31:06.280 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:06.279+0000 7f60f3222780 -1 Falling back to public interface 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:31:06.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:31:06.578 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:31:06.576+0000 7f60f3222780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:31:06.580 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:31:07.582 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:31:07.582 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:31:07.582 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:31:07.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:31:07.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:31:07.583 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:31:07.818 INFO:tasks.workunit.client.0.trial162.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2167705236,v1:127.0.0.1:6827/2167705236] [v2:127.0.0.1:6828/2167705236,v1:127.0.0.1:6829/2167705236] exists,up 5a49bb18-a42d-4fa8-a99c-537a3018a808 2026-02-13T19:31:07.819 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:31:07.819 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:31:07.819 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:31:07.819 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:252: TEST_0_osd: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-13T19:31:08.105 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:253: TEST_0_osd: ceph osd pool create ec erasure ec-profile 2026-02-13T19:31:08.480 INFO:tasks.workunit.client.0.trial162.stderr:pool 'ec' already exists 2026-02-13T19:31:08.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:255: TEST_0_osd: wait_for_clean 2026-02-13T19:31:08.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:31:08.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:31:08.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:31:08.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:31:08.491 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:31:08.491 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:31:08.491 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:31:08.491 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:31:08.491 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:31:08.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:31:08.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:31:08.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:31:08.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:31:08.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:31:08.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:31:08.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:31:08.808 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:31:08.808 INFO:tasks.workunit.client.0.trial162.stderr:2 2026-02-13T19:31:08.808 INFO:tasks.workunit.client.0.trial162.stderr:3' 2026-02-13T19:31:08.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:31:08.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:08.809 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:31:08.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T19:31:08.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T19:31:08.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T19:31:08.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:08.871 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:31:08.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-13T19:31:08.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-13T19:31:08.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-13T19:31:08.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:08.935 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:31:08.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-13T19:31:08.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-13T19:31:08.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-60129542147' 2026-02-13T19:31:08.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:08.997 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:31:09.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-13T19:31:09.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-13T19:31:09.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-60129542147 3-77309411331' 2026-02-13T19:31:09.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:09.061 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-13T19:31:09.061 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:09.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:31:09.062 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T19:31:09.062 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:09.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-13T19:31:09.064 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836484 2026-02-13T19:31:09.064 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T19:31:09.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:31:09.290 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T19:31:09.290 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:31:10.290 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:31:10.291 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:31:10.521 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T19:31:10.521 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:10.521 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-13T19:31:10.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:10.522 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:31:10.523 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-13T19:31:10.523 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:10.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-13T19:31:10.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-13T19:31:10.524 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 42949672964 2026-02-13T19:31:10.525 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:31:10.750 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-13T19:31:10.751 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:10.751 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542147 2026-02-13T19:31:10.751 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:10.752 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:31:10.752 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-13T19:31:10.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:10.753 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542147 2026-02-13T19:31:10.753 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-13T19:31:10.754 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542147 2026-02-13T19:31:10.754 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:31:10.980 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-13T19:31:10.981 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:10.981 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411331 2026-02-13T19:31:10.981 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:10.982 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:31:10.983 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411331 2026-02-13T19:31:10.983 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:10.984 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-13T19:31:10.984 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.3 seq 77309411331 2026-02-13T19:31:10.984 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411331' 2026-02-13T19:31:10.984 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:31:11.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411331 -lt 77309411331 2026-02-13T19:31:11.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:31:11.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:11.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:31:11.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:31:11.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-13T19:31:11.761 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:31:11.762 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:11.762 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:12.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-13T19:31:12.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:31:12.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:31:12.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:258: TEST_0_osd: ceph osd pool set ec min_size 3 2026-02-13T19:31:12.420 INFO:tasks.workunit.client.0.trial162.stderr:set pool 1 min_size to 3 2026-02-13T19:31:12.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:259: TEST_0_osd: wait_for_clean 2026-02-13T19:31:12.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:31:12.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:31:12.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:31:12.430 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:31:12.431 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:31:12.431 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:31:12.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:31:12.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:31:12.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:31:12.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:31:12.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:31:12.515 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:31:12.515 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:31:12.515 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:31:12.515 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:31:12.742 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:31:12.742 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:31:12.742 INFO:tasks.workunit.client.0.trial162.stderr:2 2026-02-13T19:31:12.742 INFO:tasks.workunit.client.0.trial162.stderr:3' 2026-02-13T19:31:12.742 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:31:12.742 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:12.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:31:12.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-13T19:31:12.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-13T19:31:12.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487' 2026-02-13T19:31:12.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:12.807 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:31:12.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672967 2026-02-13T19:31:12.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672967 2026-02-13T19:31:12.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-42949672967' 2026-02-13T19:31:12.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:12.872 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:31:12.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-13T19:31:12.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-13T19:31:12.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-60129542150' 2026-02-13T19:31:12.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:12.934 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:31:12.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-13T19:31:12.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-13T19:31:12.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-60129542150 3-77309411334' 2026-02-13T19:31:12.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:12.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836487 2026-02-13T19:31:12.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:12.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:31:12.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-13T19:31:12.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:13.000 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836487 2026-02-13T19:31:13.000 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836487 2026-02-13T19:31:13.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-13T19:31:13.001 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:31:13.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-13T19:31:13.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:31:14.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:31:14.231 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:31:14.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-13T19:31:14.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:14.460 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672967 2026-02-13T19:31:14.460 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:14.461 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:31:14.462 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672967 2026-02-13T19:31:14.462 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:14.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672967 2026-02-13T19:31:14.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-13T19:31:14.463 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 42949672967 2026-02-13T19:31:14.464 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:31:14.698 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-13T19:31:14.699 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:14.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542150 2026-02-13T19:31:14.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:14.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:31:14.700 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-13T19:31:14.701 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:14.701 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542150 2026-02-13T19:31:14.702 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542150 2026-02-13T19:31:14.702 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-13T19:31:14.702 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:31:14.932 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-13T19:31:14.932 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:14.933 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411334 2026-02-13T19:31:14.933 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:14.933 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:31:14.934 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411334 2026-02-13T19:31:14.934 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:14.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-13T19:31:14.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411334' 2026-02-13T19:31:14.936 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.3 seq 77309411334 2026-02-13T19:31:14.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:31:15.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-13T19:31:15.163 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:31:15.164 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:15.164 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:31:15.490 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:31:15.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-13T19:31:15.718 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:31:15.719 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:15.719 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-13T19:31:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:31:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:31:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:261: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-13T19:31:16.262 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:16.269 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:262: TEST_0_osd: ceph osd ok-to-stop 1 2026-02-13T19:31:16.495 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:16.502 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:263: TEST_0_osd: ceph osd ok-to-stop 2 2026-02-13T19:31:16.722 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:16.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:264: TEST_0_osd: ceph osd ok-to-stop 3 2026-02-13T19:31:16.947 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:16.954 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:265: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-13T19:31:17.115 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":false,"osds":[0,1],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:17.116 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T19:31:17.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:266: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-13T19:31:17.281 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":false,"osds":[2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:17.281 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T19:31:17.285 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-13T19:31:17.285 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: grep '[0]' 2026-02-13T19:31:17.512 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:17.512 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: ceph osd ok-to-stop 1 --max 2 2026-02-13T19:31:17.513 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: grep '[1]' 2026-02-13T19:31:17.741 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:17.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:271: TEST_0_osd: ceph osd pool set ec min_size 2 2026-02-13T19:31:18.056 INFO:tasks.workunit.client.0.trial162.stderr:set pool 1 min_size to 2 2026-02-13T19:31:18.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:272: TEST_0_osd: wait_for_clean 2026-02-13T19:31:18.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:31:18.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:31:18.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:31:18.072 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:31:18.073 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:31:18.073 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:31:18.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:31:18.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:31:18.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:31:18.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:31:18.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:31:18.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:31:18.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:31:18.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:31:18.155 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:31:18.384 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:31:18.384 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:31:18.384 INFO:tasks.workunit.client.0.trial162.stderr:2 2026-02-13T19:31:18.385 INFO:tasks.workunit.client.0.trial162.stderr:3' 2026-02-13T19:31:18.385 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:31:18.385 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:18.385 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:31:18.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-13T19:31:18.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-13T19:31:18.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-13T19:31:18.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:18.450 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:31:18.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-13T19:31:18.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-13T19:31:18.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672970' 2026-02-13T19:31:18.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:18.514 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:31:18.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-13T19:31:18.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-13T19:31:18.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672970 2-60129542153' 2026-02-13T19:31:18.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:18.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:31:18.640 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-13T19:31:18.640 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-13T19:31:18.641 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672970 2-60129542153 3-77309411337' 2026-02-13T19:31:18.641 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:18.641 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-13T19:31:18.641 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:18.641 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:31:18.642 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-13T19:31:18.642 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:18.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-13T19:31:18.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-13T19:31:18.643 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836490 2026-02-13T19:31:18.644 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:31:18.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T19:31:18.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:31:19.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:31:19.874 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:31:20.101 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T19:31:20.101 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:31:21.101 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:31:21.102 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:31:21.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-13T19:31:21.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:21.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672970 2026-02-13T19:31:21.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:21.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:31:21.330 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-13T19:31:21.330 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:21.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672970 2026-02-13T19:31:21.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-13T19:31:21.332 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 42949672970 2026-02-13T19:31:21.332 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:31:21.560 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-13T19:31:21.560 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:21.560 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542153 2026-02-13T19:31:21.561 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:21.561 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:31:21.562 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-13T19:31:21.562 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:21.563 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542153 2026-02-13T19:31:21.563 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-13T19:31:21.563 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542153 2026-02-13T19:31:21.564 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:31:21.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-13T19:31:21.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:21.793 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411337 2026-02-13T19:31:21.793 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:21.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:31:21.794 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411337 2026-02-13T19:31:21.794 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:21.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-13T19:31:21.795 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.3 seq 77309411337 2026-02-13T19:31:21.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411337' 2026-02-13T19:31:21.796 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:31:22.023 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-13T19:31:22.023 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:31:22.023 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:22.024 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:22.346 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-13T19:31:22.346 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:31:22.346 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:31:22.346 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:31:22.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:31:22.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:31:22.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:31:22.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:31:22.569 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-13T19:31:22.569 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:31:22.569 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:22.570 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:22.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-13T19:31:22.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:31:22.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:31:22.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:274: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-13T19:31:23.109 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:23.116 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:275: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-13T19:31:23.341 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[2,3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:23.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:276: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-13T19:31:23.505 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:23.506 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T19:31:23.509 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:277: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-13T19:31:23.670 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:23.670 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T19:31:23.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-13T19:31:23.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: grep '[0,1]' 2026-02-13T19:31:23.902 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:23.903 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: ceph osd ok-to-stop 0 --max 20 2026-02-13T19:31:23.903 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: grep '[0,1]' 2026-02-13T19:31:24.131 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:24.131 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: ceph osd ok-to-stop 2 --max 2 2026-02-13T19:31:24.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: grep '[2,3]' 2026-02-13T19:31:24.357 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:24.357 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: ceph osd ok-to-stop 2 --max 20 2026-02-13T19:31:24.358 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: grep '[2,3]' 2026-02-13T19:31:24.594 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:24.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:285: TEST_0_osd: kill_daemons td/ok-to-stop TERM osd.0 2026-02-13T19:31:24.595 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:24.596 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:24.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:24.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:24.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:24.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:24.700 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:286: TEST_0_osd: ceph osd down 0 2026-02-13T19:31:24.933 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 is already down. 2026-02-13T19:31:24.941 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:287: TEST_0_osd: wait_for_peered 2026-02-13T19:31:24.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1731: wait_for_peered: local cmd= 2026-02-13T19:31:24.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1732: wait_for_peered: local num_peered=-1 2026-02-13T19:31:24.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1733: wait_for_peered: local cur_peered 2026-02-13T19:31:24.942 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: get_timeout_delays 90 .1 2026-02-13T19:31:24.943 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:31:24.943 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:31:24.943 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:31:24.943 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:31:24.943 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:31:25.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:31:25.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: local -a delays 2026-02-13T19:31:25.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1735: wait_for_peered: local -i loop=0 2026-02-13T19:31:25.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1737: wait_for_peered: flush_pg_stats 2026-02-13T19:31:25.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:31:25.028 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:31:25.256 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:31:25.256 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:31:25.256 INFO:tasks.workunit.client.0.trial162.stderr:2 2026-02-13T19:31:25.257 INFO:tasks.workunit.client.0.trial162.stderr:3' 2026-02-13T19:31:25.257 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:31:25.257 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:25.257 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:31:25.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-13T19:31:25.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-13T19:31:25.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-13T19:31:25.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-13T19:31:25.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:25.302 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:31:25.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672973 2026-02-13T19:31:25.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672973 2026-02-13T19:31:25.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973' 2026-02-13T19:31:25.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:25.366 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:31:25.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542157 2026-02-13T19:31:25.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542157 2026-02-13T19:31:25.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973 2-60129542157' 2026-02-13T19:31:25.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:31:25.428 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:31:25.492 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411340 2026-02-13T19:31:25.492 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411340 2026-02-13T19:31:25.492 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973 2-60129542157 3-77309411340' 2026-02-13T19:31:25.492 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:25.492 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672973 2026-02-13T19:31:25.492 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:25.493 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:31:25.493 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672973 2026-02-13T19:31:25.493 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:25.494 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672973 2026-02-13T19:31:25.494 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-13T19:31:25.495 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 42949672973 2026-02-13T19:31:25.495 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:31:25.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672971 -lt 42949672973 2026-02-13T19:31:25.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:31:26.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:31:26.723 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:31:26.948 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-13T19:31:26.948 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:26.949 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542157 2026-02-13T19:31:26.949 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:26.950 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:31:26.950 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542157 2026-02-13T19:31:26.950 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:26.951 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542157 2026-02-13T19:31:26.951 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.2 seq 60129542157 2026-02-13T19:31:26.951 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-13T19:31:26.952 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:31:27.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-13T19:31:27.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:31:27.180 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411340 2026-02-13T19:31:27.180 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:31:27.181 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:31:27.181 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411340 2026-02-13T19:31:27.181 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:31:27.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411340 2026-02-13T19:31:27.182 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.3 seq 77309411340 2026-02-13T19:31:27.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411340' 2026-02-13T19:31:27.182 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:31:27.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411340 -lt 77309411340 2026-02-13T19:31:27.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: get_num_pgs 2026-02-13T19:31:27.414 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:27.414 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:27.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: test 32 == 0 2026-02-13T19:31:27.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1742: wait_for_peered: true 2026-02-13T19:31:27.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: get_num_active_or_peered 2026-02-13T19:31:27.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-13T19:31:27.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-13T19:31:27.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-13T19:31:27.742 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-13T19:31:27.742 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-13T19:31:27.963 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: cur_peered=32 2026-02-13T19:31:27.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: get_num_pgs 2026-02-13T19:31:27.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:31:27.964 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:31:28.296 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: test 32 = 32 2026-02-13T19:31:28.296 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: break 2026-02-13T19:31:28.297 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1762: wait_for_peered: return 0 2026-02-13T19:31:28.297 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:289: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-13T19:31:28.515 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":0,"num_not_ok_pgs":0} 2026-02-13T19:31:28.522 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:290: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-13T19:31:28.742 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:28.749 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:291: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-13T19:31:28.905 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:28.905 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T19:31:28.908 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:292: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-13T19:31:29.073 INFO:tasks.workunit.client.0.trial162.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-13T19:31:29.073 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-13T19:31:29.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:29.077 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:29.077 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:29.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:29.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:29.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:29.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:29.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T19:31:29.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:31:29.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:31:29.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:29.187 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:29.187 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:29.187 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:29.187 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:29.187 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:29.192 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:29.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:31:29.193 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:31:29.194 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:31:29.195 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:31:29.195 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:31:29.195 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:31:29.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:31:29.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:31:29.196 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.197 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:31:29.198 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:31:29.198 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:31:29.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:31:29.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.213 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:29.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:29.216 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:29.216 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:29.216 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:29.216 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:31:29.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:31:29.217 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:31:29.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:31:29.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:31:29.218 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:31:29.219 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.219 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:31:29.219 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:31:29.220 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.220 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:31:29.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:31:29.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:31:29.222 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:31:29.223 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.223 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.223 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:31:29.223 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:31:29.223 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:31:29.224 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T19:31:29.225 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:31:29.225 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.225 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.225 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56967 2026-02-13T19:31:29.226 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:31:29.227 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:31:29.227 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_1_mon_checks td/ok-to-stop 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:28: TEST_1_mon_checks: local dir=td/ok-to-stop 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:30: TEST_1_mon_checks: CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:32: TEST_1_mon_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T19:31:29.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T19:31:29.245 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:31:29.246 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:29.246 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:29.246 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:29.246 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.246 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.246 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:29.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T19:31:29.269 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:31:29.270 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:31:29.270 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:29.270 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:31:29.270 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:31:29.270 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:31:29.271 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:31:29.271 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:31:29.271 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:31:29.271 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:29.271 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.272 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.272 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:31:29.272 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:29.272 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get fsid 2026-02-13T19:31:29.313 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:31:29.314 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:29.314 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:31:29.314 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:31:29.314 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:31:29.314 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:31:29.314 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:31:29.314 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:31:29.315 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:29.315 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.315 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.315 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:31:29.316 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:29.316 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get mon_host 2026-02-13T19:31:29.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:34: TEST_1_mon_checks: ceph mon ok-to-stop dne 2026-02-13T19:31:29.449 INFO:tasks.workunit.client.0.trial162.stderr:invalid mon(s) specified: [dne] 2026-02-13T19:31:29.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:35: TEST_1_mon_checks: ceph mon ok-to-stop a 2026-02-13T19:31:29.531 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a] 2026-02-13T19:31:29.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:37: TEST_1_mon_checks: ceph mon ok-to-add-offline 2026-02-13T19:31:29.610 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-13T19:31:29.613 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:39: TEST_1_mon_checks: ceph mon ok-to-rm a 2026-02-13T19:31:29.687 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-13T19:31:29.690 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:40: TEST_1_mon_checks: ceph mon ok-to-rm dne 2026-02-13T19:31:29.785 INFO:tasks.workunit.client.0.trial162.stderr:mon.dne does not exist 2026-02-13T19:31:29.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:29.793 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:29.793 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:29.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:29.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:29.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:29.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:29.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T19:31:29.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:31:29.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:31:29.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:29.901 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:29.901 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:29.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:29.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:29.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:29.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:29.906 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:31:29.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:31:29.907 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:31:29.907 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:31:29.908 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:31:29.908 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:31:29.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:31:29.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:31:29.909 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.910 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:31:29.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:31:29.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:31:29.913 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:31:29.913 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.913 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.913 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:31:29.914 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:29.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:29.916 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:29.916 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:29.916 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:31:29.917 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:31:29.917 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:31:29.918 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:31:29.918 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:31:29.919 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:31:29.919 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.919 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:31:29.920 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:31:29.920 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:29.920 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:31:29.921 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:31:29.921 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:31:29.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:31:29.923 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.923 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:31:29.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:31:29.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:31:29.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T19:31:29.925 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:31:29.925 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.925 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.925 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56967 2026-02-13T19:31:29.926 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:31:29.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:31:29.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:31:29.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T19:31:29.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_2_mons_checks td/ok-to-stop 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:44: TEST_2_mons_checks: local dir=td/ok-to-stop 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:46: TEST_2_mons_checks: CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151 ' 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:48: TEST_2_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T19:31:29.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:29.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T19:31:29.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:31:29.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:31:29.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:29.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:31:29.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:31:29.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:31:29.971 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:31:29.971 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:31:29.971 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:31:29.971 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:29.971 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:29.972 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:29.972 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:31:29.972 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:29.972 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get fsid 2026-02-13T19:31:30.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:31:30.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:30.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:31:30.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:31:30.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:30.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get mon_host 2026-02-13T19:31:30.056 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:49: TEST_2_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T19:31:30.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:31:30.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:31:30.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T19:31:30.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:31:30.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T19:31:30.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T19:31:30.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:31:30.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:30.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:30.074 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:30.074 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:30.074 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:30.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:30.075 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T19:31:30.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:31:30.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T19:31:30.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:30.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:31:30.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:31:30.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:31:30.098 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:31:30.098 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:31:30.098 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:31:30.098 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:30.098 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:30.099 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:30.099 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:31:30.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:30.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get fsid 2026-02-13T19:31:30.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T19:31:30.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:30.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:31:30.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:31:30.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:30.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get mon_host 2026-02-13T19:31:30.180 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:51: TEST_2_mons_checks: ceph mon ok-to-stop dne 2026-02-13T19:31:30.275 INFO:tasks.workunit.client.0.trial162.stderr:invalid mon(s) specified: [dne] 2026-02-13T19:31:30.281 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:52: TEST_2_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:31:30.354 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (b) after stopping mons [a] 2026-02-13T19:31:30.357 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:53: TEST_2_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:31:30.431 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b] 2026-02-13T19:31:30.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:54: TEST_2_mons_checks: ceph mon ok-to-stop a b 2026-02-13T19:31:30.509 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b] 2026-02-13T19:31:30.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:56: TEST_2_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:31:30.606 INFO:tasks.workunit.client.0.trial162.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T19:31:30.613 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:58: TEST_2_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:31:30.705 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.a 2026-02-13T19:31:30.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:59: TEST_2_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:31:30.804 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.b 2026-02-13T19:31:30.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:60: TEST_2_mons_checks: ceph mon ok-to-rm dne 2026-02-13T19:31:30.904 INFO:tasks.workunit.client.0.trial162.stderr:mon.dne does not exist 2026-02-13T19:31:30.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:30.912 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:30.912 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:30.912 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:30.912 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:30.912 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:31.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:31.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:31.025 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:31.025 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:31:31.026 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:31:31.026 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:31:31.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:31:31.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:31:31.027 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:31:31.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:31.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:31:31.029 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:31:31.029 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:31.029 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:31:31.030 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:31:31.030 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:31:31.032 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:31:31.033 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.033 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:31.035 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:31.036 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:31.036 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:31.036 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:31.036 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:31.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:31:31.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:31:31.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:31:31.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:31:31.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:31:31.038 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:31:31.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:31.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:31:31.040 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:31:31.040 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:31:31.040 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:31:31.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:31:31.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:31:31.042 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:31:31.043 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.043 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.043 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:31:31.043 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:31:31.044 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:31:31.044 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T19:31:31.045 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:31:31.045 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.045 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56967 2026-02-13T19:31:31.046 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:31:31.047 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:31:31.047 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:31:31.047 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_3_mons_checks td/ok-to-stop 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:64: TEST_3_mons_checks: local dir=td/ok-to-stop 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:66: TEST_3_mons_checks: CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152 ' 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:68: TEST_3_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T19:31:31.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T19:31:31.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:31:31.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:31.065 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:31.065 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:31.065 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.065 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.065 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:31.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T19:31:31.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:31:31.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:31:31.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:31.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:31:31.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:31:31.091 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:31:31.092 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:31:31.092 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:31:31.092 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:31:31.092 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:31.092 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.092 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.092 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:31:31.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:31.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get fsid 2026-02-13T19:31:31.134 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:31.135 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.136 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.136 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:31:31.136 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:31.136 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get mon_host 2026-02-13T19:31:31.177 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:69: TEST_3_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T19:31:31.177 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:31:31.178 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:31:31.178 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T19:31:31.178 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:31:31.178 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T19:31:31.178 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T19:31:31.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:31:31.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:31.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:31.193 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:31.194 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.194 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.194 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:31.194 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T19:31:31.216 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:31:31.217 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T19:31:31.217 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:31.217 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:31:31.217 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:31:31.217 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:31:31.218 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:31:31.218 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:31:31.218 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:31:31.218 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:31.218 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.219 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.219 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:31:31.219 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:31.219 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get fsid 2026-02-13T19:31:31.262 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T19:31:31.262 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:31.262 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:31:31.262 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:31:31.263 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:31:31.263 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:31:31.263 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:31:31.263 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:31:31.264 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:31.264 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.264 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.264 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:31:31.264 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:31.264 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get mon_host 2026-02-13T19:31:31.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:70: TEST_3_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-13T19:31:31.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:31:31.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:31:31.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-13T19:31:31.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:31:31.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-13T19:31:31.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-13T19:31:31.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:31:31.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:31:31.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:31:31.322 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:31:31.322 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.322 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.322 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:31:31.322 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-13T19:31:31.344 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:31:31.345 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-13T19:31:31.345 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:31.345 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T19:31:31.345 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:31:31.345 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:31:31.346 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T19:31:31.346 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T19:31:31.346 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T19:31:31.346 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:31.346 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.347 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.347 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.c.asok 2026-02-13T19:31:31.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:31.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.c.asok config get fsid 2026-02-13T19:31:31.387 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-13T19:31:31.387 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:31:31.387 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T19:31:31.388 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:31:31.389 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.c.asok 2026-02-13T19:31:31.390 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:31:31.390 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.c.asok config get mon_host 2026-02-13T19:31:31.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:71: TEST_3_mons_checks: wait_for_quorum 60 3 2026-02-13T19:31:31.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T19:31:31.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-13T19:31:31.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T19:31:31.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-13T19:31:31.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T19:31:31.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T19:31:31.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771011151 2026-02-13T19:31:31.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:31.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011091 -lt 1771011151 ]] 2026-02-13T19:31:31.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:31:31.433 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:37.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:37.534 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:37.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:31:37.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:37.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T19:31:37.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T19:31:37.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T19:31:37.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T19:31:37.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T19:31:37.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:73: TEST_3_mons_checks: ceph mon ok-to-stop dne 2026-02-13T19:31:37.631 INFO:tasks.workunit.client.0.trial162.stderr:invalid mon(s) specified: [dne] 2026-02-13T19:31:37.637 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:74: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:31:37.731 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c) after stopping [a] 2026-02-13T19:31:37.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:75: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:31:37.832 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-13T19:31:37.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:76: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-13T19:31:37.933 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,b) after stopping [c] 2026-02-13T19:31:37.938 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:77: TEST_3_mons_checks: ceph mon ok-to-stop a b 2026-02-13T19:31:38.012 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a,b] 2026-02-13T19:31:38.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:78: TEST_3_mons_checks: ceph mon ok-to-stop b c 2026-02-13T19:31:38.089 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b,c] 2026-02-13T19:31:38.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:79: TEST_3_mons_checks: ceph mon ok-to-stop a b c 2026-02-13T19:31:38.165 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b,c] 2026-02-13T19:31:38.167 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:81: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:31:38.259 INFO:tasks.workunit.client.0.trial162.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T19:31:38.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:83: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:31:38.360 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.a 2026-02-13T19:31:38.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:84: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:31:41.462 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.b 2026-02-13T19:31:41.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:85: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-13T19:31:41.563 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.c 2026-02-13T19:31:41.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:87: TEST_3_mons_checks: kill_daemons td/ok-to-stop KILL mon.b 2026-02-13T19:31:41.569 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:31:41.569 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:31:41.569 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:31:41.570 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:31:41.570 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:31:41.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:31:41.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:88: TEST_3_mons_checks: wait_for_quorum 60 2 2026-02-13T19:31:41.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T19:31:41.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=2 2026-02-13T19:31:41.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T19:31:41.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 2 ]] 2026-02-13T19:31:41.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T19:31:41.676 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T19:31:41.677 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771011161 2026-02-13T19:31:41.677 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:41.678 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011101 -lt 1771011161 ]] 2026-02-13T19:31:41.678 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:41.678 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:44.781 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:44.781 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:44.782 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:44.782 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:44.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:44.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:44.784 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:44.785 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011104 -lt 1771011161 ]] 2026-02-13T19:31:44.785 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:44.785 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:44.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:44.884 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:44.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:44.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:44.887 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:44.887 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:44.887 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:44.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011104 -lt 1771011161 ]] 2026-02-13T19:31:44.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:44.888 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:47.989 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:47.990 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:47.990 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:47.990 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:47.992 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:47.992 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:47.992 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:47.993 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011107 -lt 1771011161 ]] 2026-02-13T19:31:47.993 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:47.993 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:48.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:48.094 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.095 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.095 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:48.096 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:48.097 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:48.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:48.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011108 -lt 1771011161 ]] 2026-02-13T19:31:48.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:48.098 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:48.197 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:48.198 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.198 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.198 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:48.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:48.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:48.200 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:48.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011108 -lt 1771011161 ]] 2026-02-13T19:31:48.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:48.202 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:48.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:48.302 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.302 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:48.303 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.304 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:48.304 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:48.304 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:48.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011108 -lt 1771011161 ]] 2026-02-13T19:31:48.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:48.306 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:48.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:48.406 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.407 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.407 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:48.409 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:48.409 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:48.409 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:48.410 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011108 -lt 1771011161 ]] 2026-02-13T19:31:48.410 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:48.410 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:48.513 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:48.513 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.514 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.514 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:48.516 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:48.516 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:48.516 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:48.517 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011108 -lt 1771011161 ]] 2026-02-13T19:31:48.517 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:48.517 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:48.615 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:48.616 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.616 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.616 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:48.618 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:48.618 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:48.618 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:48.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011108 -lt 1771011161 ]] 2026-02-13T19:31:48.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:48.619 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:48.719 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:48.719 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.720 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:48.720 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:48.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:31:48.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:31:48.722 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:31:48.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011108 -lt 1771011161 ]] 2026-02-13T19:31:48.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-13T19:31:48.723 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:31:57.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:31:57.826 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:57.827 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:31:31.057976Z","created":"2026-02-13T19:31:31.057976Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057970+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057973+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:31:31:057974+0000","crush_location":[]}]}}' 2026-02-13T19:31:57.827 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-13T19:31:57.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T19:31:57.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T19:31:57.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T19:31:57.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T19:31:57.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T19:31:57.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:90: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:32:00.903 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a] 2026-02-13T19:32:00.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:91: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:32:00.997 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-13T19:32:01.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:92: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-13T19:32:04.077 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [c] 2026-02-13T19:32:04.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:94: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:32:07.156 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-13T19:32:07.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:96: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:32:07.233 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-13T19:32:07.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:97: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:32:10.332 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.b 2026-02-13T19:32:10.337 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:98: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-13T19:32:10.410 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-13T19:32:10.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T19:32:10.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:32:10.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:32:10.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:32:10.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:32:10.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:32:10.519 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:32:10.519 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T19:32:10.519 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:32:10.519 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:32:10.519 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:32:10.520 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:32:10.520 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:32:10.520 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:32:10.520 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:32:10.520 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:32:10.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:32:10.526 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:32:10.527 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:32:10.527 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:32:10.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:32:10.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:32:10.528 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:32:10.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:10.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:32:10.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:32:10.530 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:10.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:32:10.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:32:10.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:32:10.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:32:10.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:32:10.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:32:10.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:32:10.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T19:32:10.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T19:32:10.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:32:10.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:32:10.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:32:10.539 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:32:10.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:32:10.540 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:32:10.541 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:32:10.541 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:32:10.542 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:32:10.543 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:10.543 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:32:10.543 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:32:10.543 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:10.544 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:32:10.545 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:32:10.545 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:32:10.546 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:32:10.546 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.546 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.546 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:32:10.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:32:10.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:32:10.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T19:32:10.548 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:32:10.548 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.548 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.548 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56967 2026-02-13T19:32:10.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_4_mons_checks td/ok-to-stop 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:102: TEST_4_mons_checks: local dir=td/ok-to-stop 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:104: TEST_4_mons_checks: CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153 ' 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:106: TEST_4_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T19:32:10.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T19:32:10.567 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:10.567 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:10.567 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:10.568 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:10.568 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.568 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.568 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:10.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T19:32:10.591 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:32:10.592 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:32:10.593 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:10.593 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.593 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.593 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:32:10.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:10.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get fsid 2026-02-13T19:32:10.636 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:32:10.636 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:10.636 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:32:10.636 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:10.636 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:10.636 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:32:10.637 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:32:10.637 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:32:10.638 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:10.638 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.638 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.638 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:32:10.638 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:10.638 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get mon_host 2026-02-13T19:32:10.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:107: TEST_4_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T19:32:10.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:10.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:10.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T19:32:10.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:10.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T19:32:10.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T19:32:10.698 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:10.698 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:10.698 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:10.698 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:10.698 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.698 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.698 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:10.699 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T19:32:10.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:10.722 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T19:32:10.722 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:10.722 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:32:10.722 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:10.723 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:10.723 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:32:10.723 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:32:10.723 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:32:10.724 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:10.724 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.724 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.724 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:32:10.724 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:10.724 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get fsid 2026-02-13T19:32:10.771 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T19:32:10.772 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:10.772 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:32:10.772 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:10.772 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:10.772 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:32:10.773 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:32:10.773 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:32:10.773 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:10.773 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.773 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.773 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:32:10.774 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:10.774 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get mon_host 2026-02-13T19:32:10.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:108: TEST_4_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-13T19:32:10.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:10.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:10.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-13T19:32:10.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:10.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-13T19:32:10.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-13T19:32:10.829 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:10.829 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:10.829 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:10.830 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:10.830 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.830 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.830 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:10.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-13T19:32:10.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T19:32:10.855 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T19:32:10.856 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:10.856 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.857 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.857 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.c.asok 2026-02-13T19:32:10.857 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:10.857 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.c.asok config get fsid 2026-02-13T19:32:10.899 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T19:32:10.900 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:10.901 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.901 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.901 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.c.asok 2026-02-13T19:32:10.901 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:10.901 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.c.asok config get mon_host 2026-02-13T19:32:10.943 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:109: TEST_4_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-13T19:32:10.943 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:10.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:10.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-13T19:32:10.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:10.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-13T19:32:10.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-13T19:32:10.961 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:10.961 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:10.961 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:10.961 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:10.961 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.962 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.962 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:10.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-13T19:32:10.984 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:10.985 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-13T19:32:10.985 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:10.986 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T19:32:10.986 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:10.986 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:10.986 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T19:32:10.986 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T19:32:10.986 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T19:32:10.987 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:10.988 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:10.988 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:10.988 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.d.asok 2026-02-13T19:32:10.988 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:10.988 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.d.asok config get fsid 2026-02-13T19:32:11.032 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-13T19:32:11.033 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:11.033 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T19:32:11.033 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:11.033 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:11.033 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T19:32:11.033 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T19:32:11.033 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T19:32:11.034 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:11.034 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:11.034 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:11.034 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.d.asok 2026-02-13T19:32:11.034 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:11.034 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.d.asok config get mon_host 2026-02-13T19:32:11.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:110: TEST_4_mons_checks: wait_for_quorum 60 4 2026-02-13T19:32:11.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T19:32:11.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-13T19:32:11.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T19:32:11.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-13T19:32:11.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T19:32:11.077 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T19:32:11.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771011191 2026-02-13T19:32:11.077 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:11.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011131 -lt 1771011191 ]] 2026-02-13T19:32:11.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:32:11.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:17.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:17.183 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:17.183 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:17.184 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:32:17.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T19:32:17.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T19:32:17.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T19:32:17.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T19:32:17.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T19:32:17.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:112: TEST_4_mons_checks: ceph mon ok-to-stop dne 2026-02-13T19:32:17.279 INFO:tasks.workunit.client.0.trial162.stderr:invalid mon(s) specified: [dne] 2026-02-13T19:32:17.284 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:113: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:32:17.376 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-13T19:32:17.381 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:114: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:32:20.478 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,c,d) after stopping [b] 2026-02-13T19:32:20.484 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:115: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-13T19:32:23.579 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,b,d) after stopping [c] 2026-02-13T19:32:23.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:116: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-13T19:32:23.680 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,b,c) after stopping [d] 2026-02-13T19:32:23.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:117: TEST_4_mons_checks: ceph mon ok-to-stop a b 2026-02-13T19:32:23.760 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [a,b] 2026-02-13T19:32:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:118: TEST_4_mons_checks: ceph mon ok-to-stop c d 2026-02-13T19:32:23.839 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (a,b) after stopping mons [c,d] 2026-02-13T19:32:23.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:120: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:32:23.934 INFO:tasks.workunit.client.0.trial162.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T19:32:23.940 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:122: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:32:24.036 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.a 2026-02-13T19:32:24.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:123: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:32:24.139 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.b 2026-02-13T19:32:24.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:124: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-13T19:32:24.240 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.c 2026-02-13T19:32:24.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:126: TEST_4_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-13T19:32:24.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:32:24.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:32:24.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:32:24.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:32:24.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:32:24.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:32:24.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:127: TEST_4_mons_checks: wait_for_quorum 60 3 2026-02-13T19:32:24.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T19:32:24.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-13T19:32:24.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T19:32:24.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-13T19:32:24.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T19:32:24.354 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T19:32:24.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771011204 2026-02-13T19:32:24.355 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:24.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011144 -lt 1771011204 ]] 2026-02-13T19:32:24.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:32:24.356 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:27.458 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:27.458 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:27.458 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:27.458 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:32:27.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:32:27.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:32:27.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:27.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011147 -lt 1771011204 ]] 2026-02-13T19:32:27.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:32:27.462 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:27.564 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:27.564 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:27.565 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:27.565 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:32:27.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:32:27.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:32:27.567 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:27.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011147 -lt 1771011204 ]] 2026-02-13T19:32:27.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:32:27.568 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:27.668 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:27.668 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:27.669 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:27.669 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:32:27.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:32:27.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:32:27.670 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:27.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011147 -lt 1771011204 ]] 2026-02-13T19:32:27.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:32:27.672 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:30.771 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:30.771 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:30.772 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:30.772 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:32:30.773 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:32:30.773 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:32:30.774 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:30.774 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011150 -lt 1771011204 ]] 2026-02-13T19:32:30.774 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:32:30.775 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:33.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:33.877 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:33.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:33.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:32:33.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:32:33.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:32:33.880 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:33.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011153 -lt 1771011204 ]] 2026-02-13T19:32:33.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:32:33.881 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:33.983 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:33.983 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:33.984 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:33.984 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:32:33.985 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:32:33.986 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:32:33.986 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:33.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011153 -lt 1771011204 ]] 2026-02-13T19:32:33.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:32:33.987 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:43.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:43.087 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":10,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:43.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:10.561501Z","created":"2026-02-13T19:32:10.561501Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561496+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561499+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:10:561500+0000","crush_location":[]}]}}' 2026-02-13T19:32:43.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:32:43.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T19:32:43.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T19:32:43.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T19:32:43.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T19:32:43.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T19:32:43.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:129: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:32:43.182 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-13T19:32:43.187 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:130: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:32:46.259 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-13T19:32:46.261 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:131: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-13T19:32:46.334 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-13T19:32:46.335 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:132: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-13T19:32:46.413 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-13T19:32:46.415 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:134: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:32:46.513 INFO:tasks.workunit.client.0.trial162.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T19:32:46.518 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:136: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:32:46.614 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.a 2026-02-13T19:32:46.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:137: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:32:46.715 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.b 2026-02-13T19:32:46.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:138: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-13T19:32:46.819 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.c 2026-02-13T19:32:46.824 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:139: TEST_4_mons_checks: ceph mon ok-to-rm d 2026-02-13T19:32:49.920 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.d 2026-02-13T19:32:49.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T19:32:49.928 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:32:49.928 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:32:49.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:32:49.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:32:49.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:32:50.036 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:32:50.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:32:50.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:32:50.042 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:32:50.043 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:32:50.043 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:32:50.044 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:32:50.044 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:32:50.045 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:32:50.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:50.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:32:50.046 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:32:50.046 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:50.046 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:32:50.047 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:32:50.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:32:50.054 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:32:50.054 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.054 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:32:50.056 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:32:50.056 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:32:50.056 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-13T19:32:50.056 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-13T19:32:50.056 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:32:50.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:32:50.058 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:32:50.058 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:32:50.058 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:32:50.059 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:32:50.060 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:32:50.060 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:32:50.060 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:32:50.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:50.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:32:50.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:32:50.061 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:32:50.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:32:50.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:32:50.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:32:50.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:32:50.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.064 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:32:50.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:32:50.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:32:50.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-13T19:32:50.066 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:32:50.067 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.067 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.067 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.56967 2026-02-13T19:32:50.068 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:32:50.069 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:32:50.069 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:32:50.069 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_5_mons_checks td/ok-to-stop 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:143: TEST_5_mons_checks: local dir=td/ok-to-stop 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:145: TEST_5_mons_checks: CEPH_ARGS='--fsid=cd3e6f8d-cc9e-4901-92f1-33ed193eddd9 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153,127.0.0.1:7154 ' 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:147: TEST_5_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-13T19:32:50.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-13T19:32:50.086 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:50.086 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:50.086 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:50.087 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:50.087 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.087 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:50.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-13T19:32:50.109 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:50.110 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:32:50.110 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:32:50.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:50.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:50.111 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:32:50.111 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:32:50.111 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:32:50.112 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.112 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.112 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.112 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:32:50.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get fsid 2026-02-13T19:32:50.155 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:32:50.155 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.155 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.a.asok 2026-02-13T19:32:50.156 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.157 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.a.asok config get mon_host 2026-02-13T19:32:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:148: TEST_5_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-13T19:32:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-13T19:32:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-13T19:32:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-13T19:32:50.218 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:50.218 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:50.218 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:50.219 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:50.219 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.219 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.219 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:50.219 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-13T19:32:50.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:50.243 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-13T19:32:50.243 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.243 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:32:50.243 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:50.244 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:50.244 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:32:50.244 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:32:50.244 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:32:50.244 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.245 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.245 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.245 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:32:50.245 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.245 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get fsid 2026-02-13T19:32:50.287 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-13T19:32:50.287 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.287 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-13T19:32:50.287 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:50.287 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:50.288 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-13T19:32:50.288 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-13T19:32:50.289 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-13T19:32:50.289 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.289 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.289 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.289 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.b.asok 2026-02-13T19:32:50.289 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.289 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.b.asok config get mon_host 2026-02-13T19:32:50.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:149: TEST_5_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-13T19:32:50.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:50.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:50.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-13T19:32:50.332 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:50.332 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-13T19:32:50.332 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-13T19:32:50.348 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:50.348 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:50.348 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:50.348 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:50.348 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.349 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.349 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:50.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-13T19:32:50.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T19:32:50.374 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T19:32:50.375 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.375 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.375 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.375 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.c.asok 2026-02-13T19:32:50.376 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.376 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.c.asok config get fsid 2026-02-13T19:32:50.420 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.421 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.422 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.c.asok 2026-02-13T19:32:50.422 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.422 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.c.asok config get mon_host 2026-02-13T19:32:50.465 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:150: TEST_5_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-13T19:32:50.465 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:50.465 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:50.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-13T19:32:50.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:50.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-13T19:32:50.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-13T19:32:50.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:50.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:50.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:50.482 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:50.482 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.482 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:50.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-13T19:32:50.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:50.509 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-13T19:32:50.509 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.509 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T19:32:50.510 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:50.510 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:50.510 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T19:32:50.510 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T19:32:50.510 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T19:32:50.510 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.511 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.511 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.511 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.d.asok 2026-02-13T19:32:50.511 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.511 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.d.asok config get fsid 2026-02-13T19:32:50.552 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-13T19:32:50.552 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.552 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-13T19:32:50.552 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:50.552 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.d.asok 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.553 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.d.asok config get mon_host 2026-02-13T19:32:50.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:151: TEST_5_mons_checks: run_mon td/ok-to-stop e --public-addr=127.0.0.1:7154 2026-02-13T19:32:50.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-13T19:32:50.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:32:50.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=e 2026-02-13T19:32:50.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:32:50.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/e 2026-02-13T19:32:50.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id e --mkfs --mon-data=td/ok-to-stop/e --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7154 2026-02-13T19:32:50.611 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:32:50.612 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:32:50.612 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:32:50.612 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:32:50.612 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.612 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.612 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.56967/$cluster-$name.asok' 2026-02-13T19:32:50.612 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id e --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/e '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.56967/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7154 2026-02-13T19:32:50.635 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:32:50.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e fsid 2026-02-13T19:32:50.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-13T19:32:50.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:32:50.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:32:50.637 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.e.asok 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.638 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.e.asok config get fsid 2026-02-13T19:32:50.680 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e mon_host 2026-02-13T19:32:50.680 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:32:50.680 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-13T19:32:50.681 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:32:50.681 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:32:50.681 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-13T19:32:50.681 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-13T19:32:50.681 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-13T19:32:50.682 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:32:50.682 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:32:50.682 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:32:50.682 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.56967/ceph-mon.e.asok 2026-02-13T19:32:50.682 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:32:50.682 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.56967/ceph-mon.e.asok config get mon_host 2026-02-13T19:32:50.726 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:152: TEST_5_mons_checks: wait_for_quorum 60 5 2026-02-13T19:32:50.727 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T19:32:50.727 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=5 2026-02-13T19:32:50.727 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T19:32:50.727 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 5 ]] 2026-02-13T19:32:50.727 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T19:32:50.727 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T19:32:50.728 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771011230 2026-02-13T19:32:50.728 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:32:50.728 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011170 -lt 1771011230 ]] 2026-02-13T19:32:50.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-13T19:32:50.729 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:32:56.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:32:56.834 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:32:56.835 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:32:56.835 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-13T19:32:56.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T19:32:56.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T19:32:56.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T19:32:56.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T19:32:56.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T19:32:56.838 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:154: TEST_5_mons_checks: ceph mon ok-to-stop dne 2026-02-13T19:32:56.930 INFO:tasks.workunit.client.0.trial162.stderr:invalid mon(s) specified: [dne] 2026-02-13T19:32:56.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:155: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:32:57.029 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-13T19:32:57.034 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:156: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:32:57.130 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,c,d,e) after stopping [b] 2026-02-13T19:32:57.135 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:157: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-13T19:32:57.230 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,b,d,e) after stopping [c] 2026-02-13T19:32:57.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:158: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-13T19:32:57.329 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,b,c,e) after stopping [d] 2026-02-13T19:32:57.334 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:159: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-13T19:32:57.429 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,b,c,d) after stopping [e] 2026-02-13T19:32:57.435 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:160: TEST_5_mons_checks: ceph mon ok-to-stop a b 2026-02-13T19:32:57.532 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (c,d,e) after stopping [a,b] 2026-02-13T19:32:57.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:161: TEST_5_mons_checks: ceph mon ok-to-stop c d 2026-02-13T19:32:57.632 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (a,b,e) after stopping [c,d] 2026-02-13T19:32:57.637 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:162: TEST_5_mons_checks: ceph mon ok-to-stop a b c 2026-02-13T19:32:57.712 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (d,e) after stopping mons [a,b,c] 2026-02-13T19:32:57.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:164: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:32:57.809 INFO:tasks.workunit.client.0.trial162.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T19:32:57.815 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:166: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:32:57.908 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.a 2026-02-13T19:32:57.914 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:167: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:32:58.007 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.b 2026-02-13T19:32:58.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:168: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-13T19:33:01.108 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.c 2026-02-13T19:33:01.114 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:169: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-13T19:33:01.210 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.d 2026-02-13T19:33:01.216 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:170: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-13T19:33:01.312 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.e 2026-02-13T19:33:01.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:172: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-13T19:33:01.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:01.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:01.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:01.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:01.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:173: TEST_5_mons_checks: wait_for_quorum 60 4 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T19:33:01.425 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T19:33:01.426 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771011241 2026-02-13T19:33:01.426 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:01.427 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011181 -lt 1771011241 ]] 2026-02-13T19:33:01.427 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:01.428 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:04.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:04.529 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.529 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.530 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:04.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:04.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:04.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:04.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011184 -lt 1771011241 ]] 2026-02-13T19:33:04.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:04.533 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:04.630 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:04.630 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.631 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:04.631 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.633 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:04.633 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:04.633 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:04.633 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011184 -lt 1771011241 ]] 2026-02-13T19:33:04.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:04.634 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:04.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:04.731 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:04.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.733 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:04.733 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:04.734 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:04.734 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011184 -lt 1771011241 ]] 2026-02-13T19:33:04.735 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:04.735 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:04.832 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:04.832 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:04.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:04.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:04.835 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:04.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011184 -lt 1771011241 ]] 2026-02-13T19:33:04.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:04.836 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:04.932 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:04.932 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.933 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:04.933 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:04.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:04.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:04.935 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:04.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011184 -lt 1771011241 ]] 2026-02-13T19:33:04.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:04.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:05.036 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:05.036 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:05.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:05.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:05.039 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:05.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011185 -lt 1771011241 ]] 2026-02-13T19:33:05.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:05.040 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:05.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:05.137 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:05.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.140 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:05.140 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:05.140 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:05.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011185 -lt 1771011241 ]] 2026-02-13T19:33:05.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:05.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:05.245 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:05.245 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.246 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.246 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:05.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:05.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:05.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:05.249 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011185 -lt 1771011241 ]] 2026-02-13T19:33:05.249 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:05.249 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:05.346 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:05.346 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.347 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:05.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:05.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:05.349 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:05.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011185 -lt 1771011241 ]] 2026-02-13T19:33:05.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:05.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:05.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:05.448 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.449 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.449 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:05.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:05.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:05.451 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:05.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011185 -lt 1771011241 ]] 2026-02-13T19:33:05.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:05.452 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:05.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:05.550 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.551 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.551 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:05.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:05.553 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:05.553 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:05.554 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011185 -lt 1771011241 ]] 2026-02-13T19:33:05.554 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:05.554 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:05.651 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:05.652 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.652 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:05.652 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:05.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:05.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:05.654 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:05.655 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011185 -lt 1771011241 ]] 2026-02-13T19:33:05.655 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:05.655 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:08.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:08.758 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:08.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:08.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:08.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:08.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:08.761 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:08.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011188 -lt 1771011241 ]] 2026-02-13T19:33:08.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:08.762 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:08.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:08.861 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:08.862 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:08.862 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:08.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:08.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:08.864 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:08.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011188 -lt 1771011241 ]] 2026-02-13T19:33:08.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-13T19:33:08.865 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:16.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:16.371 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:16.372 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:16.372 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-13T19:33:16.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T19:33:16.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T19:33:16.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T19:33:16.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T19:33:16.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T19:33:16.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:175: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:33:16.471 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-13T19:33:16.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:176: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:33:19.571 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (c,d,e) after stopping [b] 2026-02-13T19:33:19.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:177: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-13T19:33:19.669 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,d,e) after stopping [c] 2026-02-13T19:33:19.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:178: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-13T19:33:19.765 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,e) after stopping [d] 2026-02-13T19:33:19.770 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:179: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-13T19:33:19.862 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-13T19:33:19.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:181: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:33:19.958 INFO:tasks.workunit.client.0.trial162.stderr:adding another mon that is not yet online will not break quorum 2026-02-13T19:33:19.963 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:183: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:33:20.056 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.a 2026-02-13T19:33:20.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:184: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:33:20.155 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.b 2026-02-13T19:33:20.161 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:185: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-13T19:33:20.258 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.c 2026-02-13T19:33:20.264 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:186: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-13T19:33:23.360 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.d 2026-02-13T19:33:23.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:187: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-13T19:33:23.459 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.e 2026-02-13T19:33:23.465 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:189: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.e 2026-02-13T19:33:23.466 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:23.466 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:23.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:23.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:23.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:23.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:23.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:190: TEST_5_mons_checks: wait_for_quorum 60 3 2026-02-13T19:33:23.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-13T19:33:23.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-13T19:33:23.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-13T19:33:23.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-13T19:33:23.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-13T19:33:23.574 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-13T19:33:23.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771011263 2026-02-13T19:33:23.576 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:23.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011203 -lt 1771011263 ]] 2026-02-13T19:33:23.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:33:23.576 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:23.676 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:23.677 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:23.678 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:23.678 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:33:23.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:23.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:23.680 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:23.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011203 -lt 1771011263 ]] 2026-02-13T19:33:23.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:33:23.681 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:26.778 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:26.778 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:26.779 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:26.779 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:33:26.781 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:26.781 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:26.781 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:26.782 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011206 -lt 1771011263 ]] 2026-02-13T19:33:26.782 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:33:26.782 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:32.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:32.884 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:32.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:32.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:33:32.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:32.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:32.887 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:32.887 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011212 -lt 1771011263 ]] 2026-02-13T19:33:32.887 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:33:32.888 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:32.988 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:32.988 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:32.989 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:32.989 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:33:32.991 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:32.991 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:32.991 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:32.992 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011212 -lt 1771011263 ]] 2026-02-13T19:33:32.992 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:33:32.992 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:33.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:33.092 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:33.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:33:33.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:33.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-13T19:33:33.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-13T19:33:33.094 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-13T19:33:33.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771011213 -lt 1771011263 ]] 2026-02-13T19:33:33.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-13T19:33:33.096 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-13T19:33:39.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-13T19:33:39.197 INFO:tasks.workunit.client.0.trial162.stderr:{"election_epoch":14,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:39.197 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-13T19:33:39.198 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":14,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"cd3e6f8d-cc9e-4901-92f1-33ed193eddd9","modified":"2026-02-13T19:32:50.079942Z","created":"2026-02-13T19:32:50.079942Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079935+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079939+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079940+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-13T19:32:50:079941+0000","crush_location":[]}]}}' 2026-02-13T19:33:39.199 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-13T19:33:39.199 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-13T19:33:39.199 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-13T19:33:39.199 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-13T19:33:39.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-13T19:33:39.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:192: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-13T19:33:39.290 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-13T19:33:39.294 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:193: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-13T19:33:39.367 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-13T19:33:39.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:194: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-13T19:33:39.444 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-13T19:33:39.446 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:195: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-13T19:33:39.520 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-13T19:33:39.522 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:196: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-13T19:33:42.614 INFO:tasks.workunit.client.0.trial162.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-13T19:33:42.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:198: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-13T19:33:42.693 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-13T19:33:42.695 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:200: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-13T19:33:42.786 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.a 2026-02-13T19:33:42.791 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:201: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-13T19:33:42.865 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: removing mon.b would break quorum 2026-02-13T19:33:42.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:202: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-13T19:33:42.941 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-13T19:33:42.943 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:203: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-13T19:33:43.019 INFO:tasks.workunit.client.0.trial162.stderr:Error EBUSY: removing mon.d would break quorum 2026-02-13T19:33:43.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:204: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-13T19:33:49.115 INFO:tasks.workunit.client.0.trial162.stderr:safe to remove mon.e 2026-02-13T19:33:49.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-13T19:33:49.122 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:49.122 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:49.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:49.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:49.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:49.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:49.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-13T19:33:49.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:33:49.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:33:49.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:33:49.230 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:49.230 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:49.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:49.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:49.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:49.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:49.237 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:33:49.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:33:49.238 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:33:49.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:33:49.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:33:49.239 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:33:49.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:49.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:33:49.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:33:49.240 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:49.241 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:33:49.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:33:49.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:33:49.252 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:33:49.252 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.252 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:33:49.252 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:33:49.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:33:49.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:33:49.253 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/ok-to-stop 0 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:49.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:49.256 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:49.256 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:33:49.257 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:33:49.257 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:33:49.258 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:33:49.258 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:33:49.258 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:33:49.259 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:49.259 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:33:49.259 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:33:49.260 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:49.260 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:33:49.261 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T19:33:49.261 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-13T19:33:49.262 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:33:49.262 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.262 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.56967 2026-02-13T19:33:49.262 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.56967 2026-02-13T19:33:49.263 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:33:49.263 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:33:49.263 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-13T19:33:49.264 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T19:33:49.264 DEBUG:teuthology.orchestra.run.trial162:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T19:33:49.330 INFO:tasks.workunit:Running workunit misc/rados-striper.sh... 2026-02-13T19:33:49.330 DEBUG:teuthology.orchestra.run.trial162:workunit test misc/rados-striper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh 2026-02-13T19:33:49.395 INFO:tasks.workunit.client.0.trial162.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:33:49.400 INFO:tasks.workunit.client.0.trial162.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:33:49.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:33:49.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:33:49.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-13T19:33:49.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/rados-striper 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:20: run: local dir=td/rados-striper 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:21: run: shift 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: export CEPH_MON=127.0.0.1:7116 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: CEPH_MON=127.0.0.1:7116 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:24: run: export CEPH_ARGS 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: uuidgen 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: CEPH_ARGS+='--fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none ' 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7116 ' 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:29: run: setup td/rados-striper 2026-02-13T19:33:49.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/rados-striper 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/rados-striper 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:49.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:49.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:49.403 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:33:49.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:33:49.403 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:33:49.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:33:49.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:33:49.404 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:33:49.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:49.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:33:49.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:33:49.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:49.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:33:49.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:33:49.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-13T19:33:49.408 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:33:49.408 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.408 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:49.408 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.71693 2026-02-13T19:33:49.408 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:33:49.409 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:33:49.409 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/rados-striper 2026-02-13T19:33:49.410 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:33:49.410 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.410 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:49.410 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.71693 2026-02-13T19:33:49.411 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:33:49.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:33:49.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:33:49.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/rados-striper 1' TERM HUP INT 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:32: run: run_mon td/rados-striper a 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/rados-striper 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/rados-striper/a 2026-02-13T19:33:49.413 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/rados-striper/a --run-dir=td/rados-striper 2026-02-13T19:33:49.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:33:49.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:49.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:49.431 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:49.432 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.432 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:49.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.71693/$cluster-$name.asok' 2026-02-13T19:33:49.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/rados-striper/a '--log-file=td/rados-striper/$name.log' '--admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' --mon-cluster-log-file=td/rados-striper/log --run-dir=td/rados-striper '--pid-file=td/rados-striper/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:33:49.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:33:49.457 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:33:49.458 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:33:49.458 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.458 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:49.458 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.71693/ceph-mon.a.asok 2026-02-13T19:33:49.458 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:33:49.458 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.71693/ceph-mon.a.asok config get fsid 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:33:49.502 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:33:49.503 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:33:49.503 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.503 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:49.503 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.71693/ceph-mon.a.asok 2026-02-13T19:33:49.503 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:33:49.503 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.71693/ceph-mon.a.asok config get mon_host 2026-02-13T19:33:49.548 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:33: run: run_osd td/rados-striper 0 2026-02-13T19:33:49.548 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-13T19:33:49.548 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:33:49.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:33:49.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:33:49.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/0 2026-02-13T19:33:49.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-13T19:33:49.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:33:49.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:33:49.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/0' 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/0/journal' 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:49.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.71693/$cluster-$name.asok' 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:33:49.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:33:49.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/0 2026-02-13T19:33:49.553 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:33:49.553 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=67e1ab47-752a-4aa3-9ad4-3b14d6a8965a 2026-02-13T19:33:49.553 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 67e1ab47-752a-4aa3-9ad4-3b14d6a8965a' 2026-02-13T19:33:49.553 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 67e1ab47-752a-4aa3-9ad4-3b14d6a8965a 2026-02-13T19:33:49.554 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:33:49.555 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCdfI9pP1ARIRAAUnU7uQ5buHbU8xaP90eRxQ== 2026-02-13T19:33:49.556 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCdfI9pP1ARIRAAUnU7uQ5buHbU8xaP90eRxQ=="}' 2026-02-13T19:33:49.556 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 67e1ab47-752a-4aa3-9ad4-3b14d6a8965a -i td/rados-striper/0/new.json 2026-02-13T19:33:49.655 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:33:49.661 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/0/new.json 2026-02-13T19:33:49.662 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCdfI9pP1ARIRAAUnU7uQ5buHbU8xaP90eRxQ== --osd-uuid 67e1ab47-752a-4aa3-9ad4-3b14d6a8965a 2026-02-13T19:33:49.677 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:49.677+0000 7f08dc1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:49.679 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:49.678+0000 7f08dc1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:49.679 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:49.679+0000 7f08dc1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:49.680 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:49.680+0000 7f08dc1d8780 -1 bdev(0x3498c00 td/rados-striper/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:33:49.680 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:49.680+0000 7f08dc1d8780 -1 bluestore(td/rados-striper/0) _read_fsid unparsable uuid 2026-02-13T19:33:50.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/0/keyring 2026-02-13T19:33:50.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:33:50.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:33:50.002 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:33:50.002 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:33:50.103 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:33:50.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:33:50.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:33:50.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:33:50.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:33:50.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:33:50.116 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:50.116+0000 7f26e5c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:50.118 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:50.117+0000 7f26e5c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:50.118 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:50.118+0000 7f26e5c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:33:50.196 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:33:50.197 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:50.197 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:33:50.287 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:50.287+0000 7f26e5c33780 -1 Falling back to public interface 2026-02-13T19:33:50.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:33:50.562 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:50.562+0000 7f26e5c33780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:33:51.292 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:33:51.292 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:51.293 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:33:51.293 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:33:51.293 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:51.293 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:33:51.387 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/894801943,v1:127.0.0.1:6801/894801943] [v2:127.0.0.1:6802/894801943,v1:127.0.0.1:6803/894801943] exists,up 67e1ab47-752a-4aa3-9ad4-3b14d6a8965a 2026-02-13T19:33:51.391 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:34: run: run_osd td/rados-striper 1 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/1 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:33:51.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/1' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/1/journal' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:51.393 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.71693/$cluster-$name.asok' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:33:51.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:33:51.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:33:51.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/1 2026-02-13T19:33:51.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:33:51.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2d6a6732-9e71-487b-ba61-3dec5c471146 2026-02-13T19:33:51.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2d6a6732-9e71-487b-ba61-3dec5c471146' 2026-02-13T19:33:51.395 INFO:tasks.workunit.client.0.trial162.stdout:add osd1 2d6a6732-9e71-487b-ba61-3dec5c471146 2026-02-13T19:33:51.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:33:51.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCffI9pL3fGFxAAzIHmn9c0Zyo0AVs/ijQXoQ== 2026-02-13T19:33:51.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCffI9pL3fGFxAAzIHmn9c0Zyo0AVs/ijQXoQ=="}' 2026-02-13T19:33:51.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2d6a6732-9e71-487b-ba61-3dec5c471146 -i td/rados-striper/1/new.json 2026-02-13T19:33:51.500 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:33:51.505 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/1/new.json 2026-02-13T19:33:51.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCffI9pL3fGFxAAzIHmn9c0Zyo0AVs/ijQXoQ== --osd-uuid 2d6a6732-9e71-487b-ba61-3dec5c471146 2026-02-13T19:33:51.521 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.520+0000 7f51b9e56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:51.522 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.522+0000 7f51b9e56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:51.523 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.523+0000 7f51b9e56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:51.523 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.523+0000 7f51b9e56780 -1 bdev(0x31d8c00 td/rados-striper/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:33:51.524 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.523+0000 7f51b9e56780 -1 bluestore(td/rados-striper/1) _read_fsid unparsable uuid 2026-02-13T19:33:51.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/1/keyring 2026-02-13T19:33:51.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:33:51.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:33:51.842 INFO:tasks.workunit.client.0.trial162.stdout:adding osd1 key to auth repository 2026-02-13T19:33:51.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:33:51.942 INFO:tasks.workunit.client.0.trial162.stdout:start osd.1 2026-02-13T19:33:51.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:33:51.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:33:51.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:33:51.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:33:51.942 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:33:51.956 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.956+0000 7f3d49a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:51.958 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.957+0000 7f3d49a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:51.958 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:51.958+0000 7f3d49a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:52.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:33:52.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:33:52.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:33:52.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:33:52.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:33:52.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:52.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:33:52.039 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:33:52.040 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:52.040 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:33:52.126 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:52.126+0000 7f3d49a55780 -1 Falling back to public interface 2026-02-13T19:33:52.131 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:33:52.375 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:52.375+0000 7f3d49a55780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:33:53.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:33:53.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:53.133 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:33:53.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:33:53.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:53.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:33:53.229 INFO:tasks.workunit.client.0.trial162.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/25895217,v1:127.0.0.1:6809/25895217] [v2:127.0.0.1:6810/25895217,v1:127.0.0.1:6811/25895217] exists,up 2d6a6732-9e71-487b-ba61-3dec5c471146 2026-02-13T19:33:53.233 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:35: run: run_osd td/rados-striper 2 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/2 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-13T19:33:53.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/2' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/2/journal' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:53.235 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.71693/$cluster-$name.asok' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:33:53.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:33:53.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:33:53.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/2 2026-02-13T19:33:53.237 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:33:53.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f9d3ef5d-eef1-4bf9-a63a-402a7e0127ff 2026-02-13T19:33:53.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f9d3ef5d-eef1-4bf9-a63a-402a7e0127ff' 2026-02-13T19:33:53.237 INFO:tasks.workunit.client.0.trial162.stdout:add osd2 f9d3ef5d-eef1-4bf9-a63a-402a7e0127ff 2026-02-13T19:33:53.238 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:33:53.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQChfI9pBmpoDhAAoeUliUMRaRh3ZwaPfm8r7Q== 2026-02-13T19:33:53.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQChfI9pBmpoDhAAoeUliUMRaRh3ZwaPfm8r7Q=="}' 2026-02-13T19:33:53.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f9d3ef5d-eef1-4bf9-a63a-402a7e0127ff -i td/rados-striper/2/new.json 2026-02-13T19:33:53.343 INFO:tasks.workunit.client.0.trial162.stdout:2 2026-02-13T19:33:53.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/2/new.json 2026-02-13T19:33:53.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChfI9pBmpoDhAAoeUliUMRaRh3ZwaPfm8r7Q== --osd-uuid f9d3ef5d-eef1-4bf9-a63a-402a7e0127ff 2026-02-13T19:33:53.365 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.364+0000 7f3a9ca43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:53.366 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.366+0000 7f3a9ca43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:53.367 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.367+0000 7f3a9ca43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:53.367 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.367+0000 7f3a9ca43780 -1 bdev(0x26bec00 td/rados-striper/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:33:53.367 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.367+0000 7f3a9ca43780 -1 bluestore(td/rados-striper/2) _read_fsid unparsable uuid 2026-02-13T19:33:53.701 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/2/keyring 2026-02-13T19:33:53.701 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:33:53.702 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:33:53.702 INFO:tasks.workunit.client.0.trial162.stdout:adding osd2 key to auth repository 2026-02-13T19:33:53.702 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:33:53.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:33:53.807 INFO:tasks.workunit.client.0.trial162.stdout:start osd.2 2026-02-13T19:33:53.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=2dd9b772-dbc9-4fab-9e3e-bda65e6b2694 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.71693/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:33:53.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:33:53.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:33:53.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:33:53.821 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.821+0000 7f6ed3a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:53.822 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.822+0000 7f6ed3a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:53.823 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:53.823+0000 7f6ed3a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:53.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:33:53.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:33:53.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:33:53.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:33:53.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:33:53.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:53.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:33:53.901 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:33:53.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:53.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:33:53.992 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:33:54.000 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:54.000+0000 7f6ed3a55780 -1 Falling back to public interface 2026-02-13T19:33:54.260 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:54.260+0000 7f6ed3a55780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:33:54.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:33:54.994 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:54.994 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:33:54.995 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:33:54.995 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:54.995 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:33:55.087 INFO:tasks.workunit.client.0.trial162.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/3486071600,v1:127.0.0.1:6817/3486071600] [v2:127.0.0.1:6818/3486071600,v1:127.0.0.1:6819/3486071600] exists,up f9d3ef5d-eef1-4bf9-a63a-402a7e0127ff 2026-02-13T19:33:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:33:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:33:55.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:33:55.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:36: run: create_rbd_pool 2026-02-13T19:33:55.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:33:55.178 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:33:55.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:33:55.184 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:33:55.287 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:33:55.292 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:33:56.293 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:33:56.570 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:39: run: dd if=/dev/urandom of=td/rados-striper/toyfile bs=1234 count=1 2026-02-13T19:33:56.571 INFO:tasks.workunit.client.0.trial162.stderr:1+0 records in 2026-02-13T19:33:56.571 INFO:tasks.workunit.client.0.trial162.stderr:1+0 records out 2026-02-13T19:33:56.571 INFO:tasks.workunit.client.0.trial162.stderr:1234 bytes (1.2 kB, 1.2 KiB) copied, 9.8453e-05 s, 12.5 MB/s 2026-02-13T19:33:56.572 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:42: run: rados --pool rbd --striper put toyfile td/rados-striper/toyfile 2026-02-13T19:33:56.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: rados --pool rbd --striper stat toyfile 2026-02-13T19:33:56.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: cut -d , -f 2 2026-02-13T19:33:56.605 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-13T19:33:56.605 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: cut -d , -f 2 2026-02-13T19:33:56.620 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:47: run: echo ' size 1234' 2026-02-13T19:33:56.620 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:48: run: diff -w td/rados-striper/stripedStat td/rados-striper/refstat 2026-02-13T19:33:56.621 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:49: run: diff -w td/rados-striper/stat td/rados-striper/refstat 2026-02-13T19:33:56.621 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:50: run: rados --pool rbd stat toyfile 2026-02-13T19:33:56.636 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:51: run: grep -q 'No such file or directory' td/rados-striper/staterror 2026-02-13T19:33:56.636 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:54: run: rados --pool rbd --striper get toyfile td/rados-striper/stripedGroup 2026-02-13T19:33:56.653 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:55: run: diff -w td/rados-striper/toyfile td/rados-striper/stripedGroup 2026-02-13T19:33:56.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:56: run: rados --pool rbd get toyfile.0000000000000000 td/rados-striper/nonSTripedGroup 2026-02-13T19:33:56.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:57: run: diff -w td/rados-striper/toyfile td/rados-striper/nonSTripedGroup 2026-02-13T19:33:56.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:60: run: rados --pool rbd --striper truncate toyfile 12 2026-02-13T19:33:56.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: rados --pool rbd --striper stat toyfile 2026-02-13T19:33:56.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: cut -d , -f 2 2026-02-13T19:33:56.702 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-13T19:33:56.702 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: cut -d , -f 2 2026-02-13T19:33:56.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:63: run: echo ' size 12' 2026-02-13T19:33:56.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:64: run: diff -w td/rados-striper/stripedStat td/rados-striper/reftrunc 2026-02-13T19:33:56.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:65: run: diff -w td/rados-striper/stat td/rados-striper/reftrunc 2026-02-13T19:33:56.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:69: run: rados --pool rbd --striper setxattr toyfile somexattr somevalue 2026-02-13T19:33:56.734 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:70: run: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-13T19:33:56.749 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:71: run: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-13T19:33:56.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:72: run: echo somevalue 2026-02-13T19:33:56.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:73: run: diff -w td/rados-striper/xattrvalue td/rados-striper/refvalue 2026-02-13T19:33:56.764 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:74: run: diff -w td/rados-striper/xattrvalue2 td/rados-striper/refvalue 2026-02-13T19:33:56.764 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:75: run: rados --pool rbd --striper listxattr toyfile 2026-02-13T19:33:56.779 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:76: run: echo somexattr 2026-02-13T19:33:56.779 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:77: run: diff -w td/rados-striper/xattrlist td/rados-striper/reflist 2026-02-13T19:33:56.780 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: rados --pool rbd listxattr toyfile.0000000000000000 2026-02-13T19:33:56.780 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: grep -v striper 2026-02-13T19:33:56.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:79: run: diff -w td/rados-striper/xattrlist2 td/rados-striper/reflist 2026-02-13T19:33:56.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:80: run: rados --pool rbd --striper rmxattr toyfile somexattr 2026-02-13T19:33:56.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:82: run: local 'attr_not_found_str=No data available' 2026-02-13T19:33:56.810 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: uname 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: '[' Linux = FreeBSD ']' 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:85: run: expect_failure td/rados-striper 'No data available' rados --pool rbd --striper getxattr toyfile somexattr 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T19:33:56.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-13T19:33:56.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T19:33:56.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T19:33:56.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-13T19:33:56.828 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T19:33:56.828 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:87: run: expect_failure td/rados-striper 'No data available' rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-13T19:33:56.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T19:33:56.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T19:33:56.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-13T19:33:56.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T19:33:56.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T19:33:56.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-13T19:33:56.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T19:33:56.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T19:33:56.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-13T19:33:56.844 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T19:33:56.844 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:91: run: rados --pool rbd --striper rm toyfile 2026-02-13T19:33:56.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:92: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd --striper stat toyfile 2026-02-13T19:33:56.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T19:33:56.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T19:33:56.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-13T19:33:56.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T19:33:56.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T19:33:56.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper stat toyfile 2026-02-13T19:33:56.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T19:33:56.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T19:33:56.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:94: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd stat toyfile.0000000000000000 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T19:33:56.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd stat toyfile.0000000000000000 2026-02-13T19:33:56.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T19:33:56.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T19:33:56.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-13T19:33:56.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T19:33:56.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:98: run: teardown td/rados-striper 2026-02-13T19:33:56.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-13T19:33:56.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:33:56.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-13T19:33:56.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:56.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:56.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:56.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:56.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:56.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:56.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:33:56.999 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:33:56.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:33:57.000 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:33:57.000 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:33:57.001 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:33:57.002 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:57.002 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:33:57.002 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:33:57.002 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:57.003 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:33:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:33:57.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-13T19:33:57.010 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:33:57.010 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.010 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:57.010 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.71693 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/rados-striper 0 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:57.012 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:57.013 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:57.013 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:57.013 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:57.013 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:57.014 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:33:57.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:33:57.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:33:57.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:33:57.016 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:33:57.016 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:33:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:33:57.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:33:57.017 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:57.018 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:33:57.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T19:33:57.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-13T19:33:57.020 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:33:57.020 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.021 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.71693 2026-02-13T19:33:57.021 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.71693 2026-02-13T19:33:57.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:33:57.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:33:57.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-13T19:33:57.023 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T19:33:57.023 DEBUG:teuthology.orchestra.run.trial162:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T19:33:57.087 INFO:tasks.workunit:Running workunit misc/test-ceph-helpers.sh... 2026-02-13T19:33:57.088 DEBUG:teuthology.orchestra.run.trial162:workunit test misc/test-ceph-helpers.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh 2026-02-13T19:33:57.152 INFO:tasks.workunit.client.0.trial162.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:33:57.155 INFO:tasks.workunit.client.0.trial162.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:33:57.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: export PATH=./bin:.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:33:57.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: PATH=./bin:.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:33:57.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: export CEPH_MON=127.0.0.1:7109 2026-02-13T19:33:57.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: CEPH_MON=127.0.0.1:7109 2026-02-13T19:33:57.156 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2417: run_tests: export CEPH_ARGS 2026-02-13T19:33:57.156 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: uuidgen 2026-02-13T19:33:57.156 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: CEPH_ARGS+=' --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none ' 2026-02-13T19:33:57.157 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2419: run_tests: CEPH_ARGS+='--mon-host=127.0.0.1:7109 ' 2026-02-13T19:33:57.157 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: export CEPH_CONF=/dev/null 2026-02-13T19:33:57.157 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: CEPH_CONF=/dev/null 2026-02-13T19:33:57.157 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: set 2026-02-13T19:33:57.157 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: sed -n -e 's/^\(test_[0-9a-z_]*\) .*/\1/p' 2026-02-13T19:33:57.159 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: local 'funcs=test_activate_osd 2026-02-13T19:33:57.159 INFO:tasks.workunit.client.0.trial162.stderr:test_activate_osd_after_mark_down 2026-02-13T19:33:57.159 INFO:tasks.workunit.client.0.trial162.stderr:test_activate_osd_skip_benchmark 2026-02-13T19:33:57.159 INFO:tasks.workunit.client.0.trial162.stderr:test_destroy_osd 2026-02-13T19:33:57.159 INFO:tasks.workunit.client.0.trial162.stderr:test_display_logs 2026-02-13T19:33:57.159 INFO:tasks.workunit.client.0.trial162.stderr:test_erasure_code_plugin_exists 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_expect_failure 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_file_with_random_data 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_flush_pg_stats 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_config 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_is_making_recovery_progress 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_last_scrub_stamp 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_not_primary 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_num_active_clean 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_num_active_or_peered 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_num_pgs 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_op_scheduler 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_osd_id_used_by_pgs 2026-02-13T19:33:57.160 INFO:tasks.workunit.client.0.trial162.stderr:test_get_osds 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_get_pg 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_get_primary 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_get_timeout_delays 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_is_clean 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_kill_daemon 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_kill_daemons 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_objectstore_tool 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_pg_schedule_scrub 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_pg_scrub 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_repair 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_run_in_background 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_run_mon 2026-02-13T19:33:57.161 INFO:tasks.workunit.client.0.trial162.stderr:test_run_osd 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_set_config 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_setup 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_teardown 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_wait_background 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_wait_for_clean 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_wait_for_health_ok 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_wait_for_osd 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_wait_for_peered 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_wait_for_scrub 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:test_wait_osd_id_used_by_pgs' 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: local dir=td/ceph-helpers 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:33:57.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd td/ceph-helpers 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: test_activate_osd: local dir=td/ceph-helpers 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: test_activate_osd: setup td/ceph-helpers 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:57.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:57.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:57.165 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:33:57.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:33:57.165 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:33:57.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:33:57.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:33:57.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:33:57.167 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:33:57.167 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.167 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:57.167 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:33:57.167 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:33:57.167 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:33:57.168 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:33:57.168 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:33:57.168 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.169 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:57.169 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:33:57.170 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:33:57.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:33:57.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:33:57.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:33:57.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:33:57.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:881: test_activate_osd: run_mon td/ceph-helpers a 2026-02-13T19:33:57.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:33:57.171 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:33:57.171 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:33:57.171 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:33:57.171 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:33:57.171 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:33:57.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:33:57.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:57.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:57.188 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:57.188 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.188 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:57.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:33:57.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:33:57.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:33:57.214 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:33:57.214 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:33:57.214 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:33:57.214 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:33:57.214 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:33:57.214 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:33:57.215 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:33:57.257 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:33:57.257 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:33:57.257 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:33:57.257 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:33:57.257 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:33:57.257 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:33:57.257 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:33:57.258 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:33:57.258 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:33:57.258 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.258 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:57.258 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:33:57.258 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:33:57.258 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:33:57.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: test_activate_osd: run_mgr td/ceph-helpers x 2026-02-13T19:33:57.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:33:57.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:33:57.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:33:57.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:33:57.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:33:57.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:33:57.399 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:33:57.399 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:57.399 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:57.400 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:57.400 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.400 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:57.400 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:33:57.400 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:33:57.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:884: test_activate_osd: run_osd td/ceph-helpers 0 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:33:57.416 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:33:57.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:33:57.418 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:33:57.419 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=14fe831e-9edb-455d-94ef-d44dcc0ff50b 2026-02-13T19:33:57.419 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 14fe831e-9edb-455d-94ef-d44dcc0ff50b' 2026-02-13T19:33:57.419 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 14fe831e-9edb-455d-94ef-d44dcc0ff50b 2026-02-13T19:33:57.419 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:33:57.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQClfI9pCe2IGRAA9rVOnX0LT46Kiam6c8pXgg== 2026-02-13T19:33:57.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQClfI9pCe2IGRAA9rVOnX0LT46Kiam6c8pXgg=="}' 2026-02-13T19:33:57.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 14fe831e-9edb-455d-94ef-d44dcc0ff50b -i td/ceph-helpers/0/new.json 2026-02-13T19:33:57.534 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:33:57.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:33:57.541 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQClfI9pCe2IGRAA9rVOnX0LT46Kiam6c8pXgg== --osd-uuid 14fe831e-9edb-455d-94ef-d44dcc0ff50b 2026-02-13T19:33:57.557 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:57.556+0000 7fec9080a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:57.558 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:57.557+0000 7fec9080a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:57.559 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:57.558+0000 7fec9080a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:57.559 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:57.558+0000 7fec9080a780 -1 bdev(0x282cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:33:57.559 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:57.559+0000 7fec9080a780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:33:57.982 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:33:57.982 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:33:57.982 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:33:57.983 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:33:57.983 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:33:58.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:33:58.085 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:33:58.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:33:58.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:33:58.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:33:58.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:33:58.101 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:58.100+0000 7f29435c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:58.102 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:58.101+0000 7f29435c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:58.103 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:58.102+0000 7f29435c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:58.183 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:33:58.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:33:58.317 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:58.316+0000 7f29435c6780 -1 Falling back to public interface 2026-02-13T19:33:58.665 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:58.664+0000 7f29435c6780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:33:59.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:33:59.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:59.282 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:33:59.283 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:33:59.283 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:59.283 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:33:59.378 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1618486658,v1:127.0.0.1:6801/1618486658] [v2:127.0.0.1:6802/1618486658,v1:127.0.0.1:6803/1618486658] exists,up 14fe831e-9edb-455d-94ef-d44dcc0ff50b 2026-02-13T19:33:59.379 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: get_asok_path osd.0 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:59.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:33:59.381 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: CEPH_ARGS= 2026-02-13T19:33:59.381 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:33:59.426 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: local 'backfills={"osd_max_backfills":"1"}' 2026-02-13T19:33:59.426 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: echo '{"osd_max_backfills":"1"}' 2026-02-13T19:33:59.426 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: grep --quiet osd_max_backfills 2026-02-13T19:33:59.427 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:889: test_activate_osd: kill_daemons td/ceph-helpers TERM osd 2026-02-13T19:33:59.428 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:33:59.428 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:33:59.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:33:59.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:33:59.428 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:33:59.736 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:33:59.736 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:891: test_activate_osd: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-13T19:33:59.736 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:33:59.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:33:59.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-13T19:33:59.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:33:59.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T19:33:59.740 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:33:59.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-13T19:33:59.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T19:33:59.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T19:33:59.742 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:33:59.742 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:33:59.742 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:33:59.752 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:59.751+0000 7fd4d0e59780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:59.754 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:59.753+0000 7fd4d0e59780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:59.755 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:59.753+0000 7fd4d0e59780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:33:59.879 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:33:59.877+0000 7fd4d0e59780 -1 Falling back to public interface 2026-02-13T19:33:59.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T19:33:59.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:33:59.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:33:59.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:33:59.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:33:59.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:33:59.968 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:33:59.968 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:33:59.968 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:33:59.968 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:00.188 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:00.186+0000 7fd4d0e59780 -1 osd.0 5 log_to_monitors true 2026-02-13T19:34:00.197 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:01.198 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:01.199 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:01.199 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:01.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:01.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:01.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:01.428 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/1329035024,v1:127.0.0.1:6801/1329035024] [v2:127.0.0.1:6802/1329035024,v1:127.0.0.1:6803/1329035024] exists,up 14fe831e-9edb-455d-94ef-d44dcc0ff50b 2026-02-13T19:34:01.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:01.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:01.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:01.430 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: get_asok_path osd.0 2026-02-13T19:34:01.430 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:01.430 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:01.430 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:01.431 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.431 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.431 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:01.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: CEPH_ARGS= 2026-02-13T19:34:01.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:34:01.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: local 'backfills={"osd_max_backfills":"20"}' 2026-02-13T19:34:01.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:894: test_activate_osd: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-13T19:34:01.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:896: test_activate_osd: teardown td/ceph-helpers 2026-02-13T19:34:01.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:01.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:01.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:01.478 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:01.478 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:01.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:01.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:01.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:01.584 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:01.584 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:01.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:01.585 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:01.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:01.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:01.587 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:01.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:01.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:01.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:01.588 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:01.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:01.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:01.590 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:01.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:01.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.594 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.594 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:01.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_after_mark_down td/ceph-helpers 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:900: test_activate_osd_after_mark_down: local dir=td/ceph-helpers 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:902: test_activate_osd_after_mark_down: setup td/ceph-helpers 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:01.596 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:01.597 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:01.597 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:01.597 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:01.597 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:01.597 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:01.597 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:01.598 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:01.598 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:01.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:01.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:01.599 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:01.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:01.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:01.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:01.600 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:01.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:01.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:01.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:01.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:01.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.603 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:01.604 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:01.604 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:01.604 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:01.605 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:01.605 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.605 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.605 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:01.606 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:01.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:01.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:01.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:01.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:904: test_activate_osd_after_mark_down: run_mon td/ceph-helpers a 2026-02-13T19:34:01.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:01.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:01.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:01.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:01.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:01.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:34:01.626 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:01.626 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:01.626 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:01.626 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:01.626 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.626 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.626 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:01.627 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:34:01.651 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:01.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:01.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:01.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:01.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:01.652 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:01.696 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:01.697 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:01.697 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:01.697 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:01.697 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:01.697 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:01.697 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:01.697 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:01.698 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:01.698 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.698 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.698 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:01.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:01.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:01.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:905: test_activate_osd_after_mark_down: run_mgr td/ceph-helpers x 2026-02-13T19:34:01.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:01.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:01.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:01.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:01.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:01.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:01.841 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:01.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:01.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:01.842 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:01.842 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.842 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:01.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:01.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:01.861 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:907: test_activate_osd_after_mark_down: run_osd td/ceph-helpers 0 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:01.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:01.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:01.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:01.865 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:01.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0f19f7c3-8827-4d88-a012-15d505fabd38 2026-02-13T19:34:01.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0f19f7c3-8827-4d88-a012-15d505fabd38' 2026-02-13T19:34:01.865 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 0f19f7c3-8827-4d88-a012-15d505fabd38 2026-02-13T19:34:01.865 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:01.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCpfI9pIW7hMxAAHIE3y6YbCmfuza0NSwxRGA== 2026-02-13T19:34:01.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCpfI9pIW7hMxAAHIE3y6YbCmfuza0NSwxRGA=="}' 2026-02-13T19:34:01.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0f19f7c3-8827-4d88-a012-15d505fabd38 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:01.977 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:01.985 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:01.986 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCpfI9pIW7hMxAAHIE3y6YbCmfuza0NSwxRGA== --osd-uuid 0f19f7c3-8827-4d88-a012-15d505fabd38 2026-02-13T19:34:02.002 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.001+0000 7fbb8422d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:02.003 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.002+0000 7fbb8422d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:02.004 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.003+0000 7fbb8422d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:02.005 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.003+0000 7fbb8422d780 -1 bdev(0x3f74c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:02.005 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.003+0000 7fbb8422d780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:02.435 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:02.435 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:02.436 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:02.436 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:02.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:02.542 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:02.543 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:02.543 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:02.544 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:02.544 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:02.544 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:02.558 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.557+0000 7f758f60f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:02.559 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.558+0000 7f758f60f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:02.560 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.559+0000 7f758f60f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:02.641 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:02.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:02.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:02.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:02.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:02.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:02.643 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:02.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:02.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:02.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:02.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:02.792 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:02.790+0000 7f758f60f780 -1 Falling back to public interface 2026-02-13T19:34:03.085 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:03.083+0000 7f758f60f780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:03.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:03.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:03.741 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:03.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:03.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:03.741 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:03.843 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/153943262,v1:127.0.0.1:6801/153943262] [v2:127.0.0.1:6802/153943262,v1:127.0.0.1:6803/153943262] exists,up 0f19f7c3-8827-4d88-a012-15d505fabd38 2026-02-13T19:34:03.844 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:03.844 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:03.844 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:03.844 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-13T19:34:03.844 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:03.844 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:03.844 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:03.845 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:03.845 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:03.845 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:03.845 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-13T19:34:03.845 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:34:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"1"}' 2026-02-13T19:34:03.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: echo '{"osd_max_backfills":"1"}' 2026-02-13T19:34:03.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: grep --quiet osd_max_backfills 2026-02-13T19:34:03.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:912: test_activate_osd_after_mark_down: kill_daemons td/ceph-helpers TERM osd 2026-02-13T19:34:03.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:03.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:03.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:03.895 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:03.895 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:04.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:04.200 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:913: test_activate_osd_after_mark_down: ceph osd down 0 2026-02-13T19:34:04.429 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 is already down. 2026-02-13T19:34:04.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:914: test_activate_osd_after_mark_down: wait_for_osd down 0 2026-02-13T19:34:04.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-13T19:34:04.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:04.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:04.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:04.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:04.438 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:04.439 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:04.439 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:04.439 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-13T19:34:04.667 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/153943262,v1:127.0.0.1:6801/153943262] [v2:127.0.0.1:6802/153943262,v1:127.0.0.1:6803/153943262] exists 0f19f7c3-8827-4d88-a012-15d505fabd38 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: test_activate_osd_after_mark_down: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:34:04.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:04.672 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:04.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:34:04.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-13T19:34:04.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:04.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T19:34:04.674 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:04.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-13T19:34:04.675 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T19:34:04.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T19:34:04.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:34:04.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:34:04.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:34:04.685 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:04.684+0000 7f497d015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:04.686 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:04.685+0000 7f497d015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:04.687 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:04.686+0000 7f497d015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:04.817 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:04.815+0000 7f497d015780 -1 Falling back to public interface 2026-02-13T19:34:04.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T19:34:04.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:04.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:04.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:04.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:04.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:04.902 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:04.902 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:04.903 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:04.903 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:05.131 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:05.130+0000 7f497d015780 -1 osd.0 5 log_to_monitors true 2026-02-13T19:34:05.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:06.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:06.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:06.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:06.137 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:06.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:06.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:06.370 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/2934567768,v1:127.0.0.1:6801/2934567768] [v2:127.0.0.1:6802/2934567768,v1:127.0.0.1:6803/2934567768] exists,up 0f19f7c3-8827-4d88-a012-15d505fabd38 2026-02-13T19:34:06.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:06.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:06.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:06.372 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-13T19:34:06.372 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:06.372 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:06.373 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:06.373 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.373 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.373 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:06.373 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-13T19:34:06.373 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_max_backfills 2026-02-13T19:34:06.420 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"20"}' 2026-02-13T19:34:06.420 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: test_activate_osd_after_mark_down: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-13T19:34:06.420 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: test_activate_osd_after_mark_down: teardown td/ceph-helpers 2026-02-13T19:34:06.420 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:06.421 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:06.421 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:06.421 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:06.421 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:06.421 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:06.421 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:06.421 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:06.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:06.526 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:06.527 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:06.528 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:06.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:06.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:06.530 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:06.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:06.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:06.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:06.531 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:06.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:06.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:06.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:06.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:06.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:06.538 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_skip_benchmark td/ceph-helpers 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: test_activate_osd_skip_benchmark: local dir=td/ceph-helpers 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:927: test_activate_osd_skip_benchmark: setup td/ceph-helpers 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:06.539 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:06.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:06.540 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:06.540 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:06.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:06.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:06.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:06.541 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:06.541 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:06.541 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:06.541 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:06.542 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:06.542 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:06.542 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:06.543 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:06.543 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:06.543 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:06.543 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:06.544 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:06.544 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:06.545 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:06.546 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:06.546 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.546 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.546 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:06.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:06.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:06.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:06.548 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:06.548 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.548 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.548 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: test_activate_osd_skip_benchmark: run_mon td/ceph-helpers a 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:06.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:06.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:06.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:06.551 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:34:06.567 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:06.567 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:06.567 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:06.567 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:06.568 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.568 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.568 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:06.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:34:06.591 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:06.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:06.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:06.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:06.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:06.593 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:06.594 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:06.636 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:06.636 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.637 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:06.638 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:06.638 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:06.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:930: test_activate_osd_skip_benchmark: run_mgr td/ceph-helpers x 2026-02-13T19:34:06.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:06.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:06.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:06.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:06.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:06.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:06.776 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:06.776 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:06.776 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:06.776 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:06.776 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.777 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.777 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:06.777 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:06.778 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:933: test_activate_osd_skip_benchmark: run_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:06.795 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:06.796 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-13T19:34:06.797 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:06.797 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:06.797 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c 2026-02-13T19:34:06.797 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c' 2026-02-13T19:34:06.797 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c 2026-02-13T19:34:06.797 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:06.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCufI9plxkdMBAAKZDEY32U6TtsOt/+YWfjew== 2026-02-13T19:34:06.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCufI9plxkdMBAAKZDEY32U6TtsOt/+YWfjew=="}' 2026-02-13T19:34:06.807 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c -i td/ceph-helpers/0/new.json 2026-02-13T19:34:06.913 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:06.919 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:06.919 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true --mkfs --key AQCufI9plxkdMBAAKZDEY32U6TtsOt/+YWfjew== --osd-uuid f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c 2026-02-13T19:34:06.936 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:06.935+0000 7fe02b84f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:06.937 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:06.936+0000 7fe02b84f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:06.938 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:06.937+0000 7fe02b84f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:06.939 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:06.937+0000 7fe02b84f780 -1 bdev(0x3892c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:06.939 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:06.938+0000 7fe02b84f780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:07.325 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:07.325 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:07.326 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:07.326 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:07.327 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:07.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:07.432 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:07.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T19:34:07.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:07.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:07.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:07.447 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:07.446+0000 7f761605c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:07.448 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:07.447+0000 7f761605c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:07.449 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:07.448+0000 7f761605c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:07.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:07.629 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:07.657 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:07.656+0000 7f761605c780 -1 Falling back to public interface 2026-02-13T19:34:08.060 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:08.059+0000 7f761605c780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:08.630 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:08.630 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:08.630 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:08.631 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:08.631 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:08.631 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:08.727 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1076236082,v1:127.0.0.1:6801/1076236082] [v2:127.0.0.1:6802/1076236082,v1:127.0.0.1:6803/1076236082] exists,up f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c 2026-02-13T19:34:08.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:08.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:08.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T19:34:08.730 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-13T19:34:08.775 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: local 'max_iops_hdd_def={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-13T19:34:08.777 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T19:34:08.777 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:08.777 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:08.777 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:08.777 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:08.777 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:08.778 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:08.778 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T19:34:08.778 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-13T19:34:08.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: local 'max_iops_ssd_def={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-13T19:34:08.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:940: test_activate_osd_skip_benchmark: kill_daemons td/ceph-helpers TERM osd 2026-02-13T19:34:08.827 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:08.827 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:08.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:08.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:08.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:08.931 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:08.932 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:941: test_activate_osd_skip_benchmark: ceph osd down 0 2026-02-13T19:34:09.159 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 is already down. 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:942: test_activate_osd_skip_benchmark: wait_for_osd down 0 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:09.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-13T19:34:09.393 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/1076236082,v1:127.0.0.1:6801/1076236082] [v2:127.0.0.1:6802/1076236082,v1:127.0.0.1:6803/1076236082] exists f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c 2026-02-13T19:34:09.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:946: test_activate_osd_skip_benchmark: activate_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:09.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:09.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T19:34:09.398 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:09.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-13T19:34:09.399 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T19:34:09.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T19:34:09.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:34:09.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:34:09.399 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:34:09.410 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:09.409+0000 7fb8901fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:09.412 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:09.411+0000 7fb8901fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:09.413 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:09.412+0000 7fb8901fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:09.541 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:09.539+0000 7fb8901fd780 -1 Falling back to public interface 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:09.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:09.847 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:09.846+0000 7fb8901fd780 -1 osd.0 5 log_to_monitors true 2026-02-13T19:34:09.857 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:10.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:10.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:10.859 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:10.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:10.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:10.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:11.089 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/446846826,v1:127.0.0.1:6801/446846826] [v2:127.0.0.1:6802/446846826,v1:127.0.0.1:6803/446846826] exists,up f0ac2ea5-a331-4b96-b2fe-07f5164cfe5c 2026-02-13T19:34:11.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:11.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:11.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:11.090 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T19:34:11.090 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:11.090 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:11.091 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:11.091 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.091 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.091 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:11.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T19:34:11.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-13T19:34:11.139 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: local 'max_iops_hdd_after_boot={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-13T19:34:11.140 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-13T19:34:11.140 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:11.140 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:11.140 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:11.140 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.140 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.141 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:11.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-13T19:34:11.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-13T19:34:11.188 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: local 'max_iops_ssd_after_boot={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:953: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' = '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:954: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' = '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:956: test_activate_osd_skip_benchmark: teardown td/ceph-helpers 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:11.189 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:11.295 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:11.295 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:11.296 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:11.297 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:11.298 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:11.298 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:11.298 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:11.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:11.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:11.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:11.300 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:11.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:11.302 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:11.302 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:11.305 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:11.305 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.305 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:11.307 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:11.307 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:11.307 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:11.307 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_destroy_osd td/ceph-helpers 2026-02-13T19:34:11.307 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:787: test_destroy_osd: local dir=td/ceph-helpers 2026-02-13T19:34:11.307 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:789: test_destroy_osd: setup td/ceph-helpers 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:11.308 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:11.309 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:11.309 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:11.309 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:11.309 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:11.310 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:11.310 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:11.310 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:11.311 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:11.311 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:11.312 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:11.312 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:11.312 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:11.313 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:11.313 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:11.314 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:11.315 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.315 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:11.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:11.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:11.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:11.317 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:11.317 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.317 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:790: test_destroy_osd: run_mon td/ceph-helpers a 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:11.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:11.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:34:11.336 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:11.336 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:11.336 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:11.337 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:11.337 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.337 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.337 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:11.337 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:34:11.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:11.361 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:11.361 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:11.361 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:11.361 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:11.362 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:11.363 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:11.405 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:11.406 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:11.406 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.406 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.406 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:11.406 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:11.406 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:11.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:791: test_destroy_osd: run_mgr td/ceph-helpers x 2026-02-13T19:34:11.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:11.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:11.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:11.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:11.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:11.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:11.556 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:11.556 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:11.556 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:11.556 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:11.556 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.556 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.556 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:11.557 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:11.557 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:11.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:792: test_destroy_osd: run_osd td/ceph-helpers 0 2026-02-13T19:34:11.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:11.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:11.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:11.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:11.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0954a193-349f-4b68-be1f-238bce0cc4b8 2026-02-13T19:34:11.579 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 0954a193-349f-4b68-be1f-238bce0cc4b8 2026-02-13T19:34:11.579 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0954a193-349f-4b68-be1f-238bce0cc4b8' 2026-02-13T19:34:11.579 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:11.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCzfI9pY0fkIhAAHVXtVWzFkz5iLwX03aDHhg== 2026-02-13T19:34:11.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCzfI9pY0fkIhAAHVXtVWzFkz5iLwX03aDHhg=="}' 2026-02-13T19:34:11.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0954a193-349f-4b68-be1f-238bce0cc4b8 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:11.688 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:11.695 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:11.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCzfI9pY0fkIhAAHVXtVWzFkz5iLwX03aDHhg== --osd-uuid 0954a193-349f-4b68-be1f-238bce0cc4b8 2026-02-13T19:34:11.712 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:11.711+0000 7faaefc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:11.713 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:11.712+0000 7faaefc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:11.714 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:11.713+0000 7faaefc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:11.715 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:11.713+0000 7faaefc0e780 -1 bdev(0x2dbcc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:11.715 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:11.714+0000 7faaefc0e780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:12.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:12.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:12.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:12.133 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:12.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:12.237 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:12.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:12.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:12.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:12.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:12.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:12.252 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:12.251+0000 7f5331c52780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:12.254 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:12.252+0000 7f5331c52780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:12.254 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:12.253+0000 7f5331c52780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:12.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:12.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:12.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:12.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:12.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:12.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:12.337 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:12.337 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:12.338 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:12.338 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:12.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:12.469 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:12.468+0000 7f5331c52780 -1 Falling back to public interface 2026-02-13T19:34:12.874 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:12.873+0000 7f5331c52780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:13.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:13.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:13.432 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:13.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:13.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:13.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:13.531 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3964871760,v1:127.0.0.1:6801/3964871760] [v2:127.0.0.1:6802/3964871760,v1:127.0.0.1:6803/3964871760] exists,up 0954a193-349f-4b68-be1f-238bce0cc4b8 2026-02-13T19:34:13.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:13.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:13.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:13.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:793: test_destroy_osd: destroy_osd td/ceph-helpers 0 2026-02-13T19:34:13.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/ceph-helpers 2026-02-13T19:34:13.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=0 2026-02-13T19:34:13.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.0 2026-02-13T19:34:13.686 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 is already out. 2026-02-13T19:34:13.692 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/ceph-helpers TERM osd.0 2026-02-13T19:34:13.693 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:13.693 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:13.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:13.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:13.693 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:13.999 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:14.000 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.0 2026-02-13T19:34:14.230 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 is already down. 2026-02-13T19:34:14.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.0 --yes-i-really-mean-it 2026-02-13T19:34:14.460 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 does not exist 2026-02-13T19:34:14.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/ceph-helpers/0 2026-02-13T19:34:14.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers/0 2026-02-13T19:34:14.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:14.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers/0 KILL 2026-02-13T19:34:14.468 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:14.468 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:14.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:14.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:14.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:14.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:14.470 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:14.470 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:14.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:14.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:14.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:14.472 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:14.473 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:14.473 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:14.473 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:14.473 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:14.474 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:14.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:14.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers/0 2026-02-13T19:34:14.477 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:14.477 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:14.477 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:14.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:14.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:14.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:14.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/ceph-helpers/0 2026-02-13T19:34:14.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: ceph osd dump 2026-02-13T19:34:14.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: grep 'osd. ' 2026-02-13T19:34:14.703 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:795: test_destroy_osd: teardown td/ceph-helpers 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:14.704 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:14.810 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:14.810 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:14.811 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:14.811 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:14.812 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:14.813 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:14.813 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:14.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:14.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:14.814 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:14.815 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:14.816 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:14.817 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:14.817 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:14.819 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:14.819 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:14.820 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:14.820 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_display_logs td/ceph-helpers 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2158: test_display_logs: local dir=td/ceph-helpers 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2160: test_display_logs: setup td/ceph-helpers 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:14.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:14.823 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:14.823 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:14.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:14.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:14.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:14.824 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:14.824 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:14.824 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:14.824 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:14.825 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:14.825 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:14.825 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:14.825 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:14.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:14.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:14.826 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:14.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:14.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:14.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:14.829 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:14.829 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:14.829 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:14.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:14.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:14.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:14.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:14.831 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:14.831 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:14.831 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:14.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2161: test_display_logs: run_mon td/ceph-helpers a 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:14.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:14.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:14.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:14.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:34:14.851 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:14.851 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:14.852 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:14.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:14.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:14.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:14.852 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:14.852 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:34:14.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:14.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:14.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:14.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:14.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:14.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:14.877 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:14.877 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:14.877 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:14.878 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:14.878 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:14.878 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:14.878 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:14.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:14.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:14.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:14.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:14.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:14.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:14.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:14.922 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:14.922 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:14.923 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:14.923 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:14.923 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:14.923 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:14.923 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:14.923 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:14.924 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:14.965 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2162: test_display_logs: kill_daemons td/ceph-helpers 2026-02-13T19:34:14.965 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:14.965 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:14.966 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:14.966 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:14.966 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.071 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.071 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2163: test_display_logs: display_logs td/ceph-helpers 2026-02-13T19:34:15.071 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2148: display_logs: local dir=td/ceph-helpers 2026-02-13T19:34:15.071 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: display_logs: find td/ceph-helpers -maxdepth 1 -name '*.log' 2026-02-13T19:34:15.071 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-13T19:34:15.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2152: display_logs: echo '======================= td/ceph-helpers/mon.a.log' 2026-02-13T19:34:15.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2153: display_logs: cat td/ceph-helpers/mon.a.log 2026-02-13T19:34:15.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-13T19:34:15.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2164: test_display_logs: grep --quiet mon.a.log td/ceph-helpers/log.out 2026-02-13T19:34:15.075 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2165: test_display_logs: teardown td/ceph-helpers 2026-02-13T19:34:15.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.080 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.082 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:15.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_erasure_code_plugin_exists td/ceph-helpers 2026-02-13T19:34:15.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: test_erasure_code_plugin_exists: local dir=td/ceph-helpers 2026-02-13T19:34:15.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: test_erasure_code_plugin_exists: setup td/ceph-helpers 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.089 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.091 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.091 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.094 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.095 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.095 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.096 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:15.096 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:15.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.097 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:15.098 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: test_erasure_code_plugin_exists: run_mon td/ceph-helpers a 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:15.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:34:15.117 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:15.117 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:15.117 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:15.117 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:15.117 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.117 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.118 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:15.118 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:34:15.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:15.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:15.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:15.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:15.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:15.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:15.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:15.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:15.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:15.144 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:15.144 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.144 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.144 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:15.144 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:15.144 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:15.186 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:15.186 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:15.186 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:15.186 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:15.186 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:15.187 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:15.187 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:15.187 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:15.188 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:15.188 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.188 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.188 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:15.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:15.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:15.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: test_erasure_code_plugin_exists: run_mgr td/ceph-helpers x 2026-02-13T19:34:15.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:15.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:15.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:15.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:15.233 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:15.233 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:15.331 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:15.331 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:15.331 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:15.331 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:15.331 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.331 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.332 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:15.332 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:15.332 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:15.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: test_erasure_code_plugin_exists: erasure_code_plugin_exists jerasure 2026-02-13T19:34:15.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=jerasure 2026-02-13T19:34:15.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-13T19:34:15.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-13T19:34:15.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-13T19:34:15.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-13T19:34:15.349 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-13T19:34:15.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='jerasure.*No such file' 2026-02-13T19:34:15.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=jerasure 2026-02-13T19:34:15.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-13T19:34:15.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-13T19:34:15.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-13T19:34:15.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-13T19:34:15.631 INFO:tasks.workunit.client.0.trial162.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-13T19:34:15.638 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-13T19:34:15.639 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: test_erasure_code_plugin_exists: erasure_code_plugin_exists FAKE 2026-02-13T19:34:15.639 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=FAKE 2026-02-13T19:34:15.639 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-13T19:34:15.639 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-13T19:34:15.639 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-13T19:34:15.639 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-13T19:34:15.639 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-13T19:34:15.640 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='FAKE.*No such file' 2026-02-13T19:34:15.640 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=FAKE 2026-02-13T19:34:15.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s='Error EIO: load dlopen(/usr/lib64/ceph/erasure-code/libec_FAKE.so): /usr/lib64/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory' 2026-02-13T19:34:15.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=5 2026-02-13T19:34:15.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 5 -eq 0 ']' 2026-02-13T19:34:15.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: echo Error EIO: load 'dlopen(/usr/lib64/ceph/erasure-code/libec_FAKE.so):' /usr/lib64/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory 2026-02-13T19:34:15.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: grep --quiet 'FAKE.*No such file' 2026-02-13T19:34:15.724 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2119: erasure_code_plugin_exists: status=1 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 1 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: test_erasure_code_plugin_exists: teardown td/ceph-helpers 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.831 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.832 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.832 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.834 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.836 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.839 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.839 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.839 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.839 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:15.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_expect_failure td/ceph-helpers 2026-02-13T19:34:15.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: test_expect_failure: local dir=td/ceph-helpers 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2039: test_expect_failure: setup td/ceph-helpers 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.843 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.844 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.844 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.845 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.845 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.845 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.845 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.846 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.846 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.846 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.847 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.847 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.849 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.849 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.849 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.849 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.849 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.850 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.850 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:15.851 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:15.851 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.851 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.851 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:15.852 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:15.853 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:15.853 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:15.853 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2040: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 1' 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 1' 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T19:34:15.854 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2042: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 0' 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 0' 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2023: expect_failure: success=true 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: true 2026-02-13T19:34:15.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:FAIL 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2043: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo UNEXPECTED ; exit 1' 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-13T19:34:15.858 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-13T19:34:15.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo UNEXPECTED ; exit 1' 2026-02-13T19:34:15.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-13T19:34:15.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-13T19:34:15.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T19:34:15.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-13T19:34:15.860 INFO:tasks.workunit.client.0.trial162.stderr:UNEXPECTED 2026-02-13T19:34:15.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-13T19:34:15.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: test_expect_failure: teardown td/ceph-helpers 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.863 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.863 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.864 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.866 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.866 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.867 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.870 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.870 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.870 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.870 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_file_with_random_data td/ceph-helpers 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2350: test_file_with_random_data: local dir=td/ceph-helpers 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2352: test_file_with_random_data: setup td/ceph-helpers 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.873 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.873 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.874 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.874 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.875 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.877 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.877 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.879 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.879 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.880 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:15.881 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:15.882 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.882 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:15.882 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:15.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:15.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:15.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:15.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: file_with_random_data 4000 2026-02-13T19:34:15.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2343: file_with_random_data: local size_bytes=4000 2026-02-13T19:34:15.884 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: mktemp 2026-02-13T19:34:15.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local file=/tmp/tmp.ZbcAHwWLU4 2026-02-13T19:34:15.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.ZbcAHwWLU4 bs=4000 count=1 2026-02-13T19:34:15.885 INFO:tasks.workunit.client.0.trial162.stderr:1+0 records in 2026-02-13T19:34:15.886 INFO:tasks.workunit.client.0.trial162.stderr:1+0 records out 2026-02-13T19:34:15.886 INFO:tasks.workunit.client.0.trial162.stderr:4000 bytes (4.0 kB, 3.9 KiB) copied, 5.7947e-05 s, 69.0 MB/s 2026-02-13T19:34:15.886 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: printf %s /tmp/tmp.ZbcAHwWLU4 2026-02-13T19:34:15.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: local file=/tmp/tmp.ZbcAHwWLU4 2026-02-13T19:34:15.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2354: test_file_with_random_data: test -f /tmp/tmp.ZbcAHwWLU4 2026-02-13T19:34:15.886 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: stat -c %s /tmp/tmp.ZbcAHwWLU4 2026-02-13T19:34:15.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: test 4000 = 4000 2026-02-13T19:34:15.887 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2356: test_file_with_random_data: rm /tmp/tmp.ZbcAHwWLU4 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2357: test_file_with_random_data: teardown td/ceph-helpers 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.890 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.890 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.896 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.896 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.896 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.896 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_flush_pg_stats td/ceph-helpers 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2280: test_flush_pg_stats: local dir=td/ceph-helpers 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: test_flush_pg_stats: setup td/ceph-helpers 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:15.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:15.899 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:15.899 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:15.899 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:15.899 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:15.899 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:15.899 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:15.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:15.900 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:15.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:15.900 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:15.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:15.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:15.901 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:15.902 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.902 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:15.902 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:15.902 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:15.903 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:15.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:15.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:15.905 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:15.905 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.905 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:15.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:15.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:15.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:15.907 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:15.907 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.907 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.907 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:15.908 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:15.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:15.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:15.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:15.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2283: test_flush_pg_stats: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:15.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:15.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:15.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:15.910 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:15.910 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:15.910 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:15.927 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:15.927 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:15.927 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:15.927 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:15.927 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.927 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.927 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:15.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:15.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:15.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:15.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:15.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:15.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:15.953 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:15.955 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:15.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:15.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:15.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:15.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:15.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:15.999 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:15.999 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:15.999 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:16.000 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:16.000 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:16.000 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:16.000 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:16.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:16.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: test_flush_pg_stats: run_mgr td/ceph-helpers x 2026-02-13T19:34:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:16.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:16.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:16.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:16.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:16.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:16.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:16.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:16.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:16.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:16.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:16.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:16.144 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:16.144 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:16.161 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: test_flush_pg_stats: run_osd td/ceph-helpers 0 2026-02-13T19:34:16.161 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:16.161 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:16.162 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:16.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:16.164 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:16.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bbeaefda-fd00-49e3-a19f-72e2ce0509b9 2026-02-13T19:34:16.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bbeaefda-fd00-49e3-a19f-72e2ce0509b9' 2026-02-13T19:34:16.165 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 bbeaefda-fd00-49e3-a19f-72e2ce0509b9 2026-02-13T19:34:16.165 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:16.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC4fI9p8ChiChAAU2a4aK82ZoR4KASpUae8dQ== 2026-02-13T19:34:16.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC4fI9p8ChiChAAU2a4aK82ZoR4KASpUae8dQ=="}' 2026-02-13T19:34:16.175 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bbeaefda-fd00-49e3-a19f-72e2ce0509b9 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:16.281 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:16.288 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:16.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC4fI9p8ChiChAAU2a4aK82ZoR4KASpUae8dQ== --osd-uuid bbeaefda-fd00-49e3-a19f-72e2ce0509b9 2026-02-13T19:34:16.305 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.304+0000 7f7679a3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:16.307 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.306+0000 7f7679a3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:16.308 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.307+0000 7f7679a3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:16.308 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.307+0000 7f7679a3b780 -1 bdev(0x26e0c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:16.308 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.307+0000 7f7679a3b780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:16.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:16.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:16.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:16.718 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:16.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:16.822 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:16.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:16.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:16.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:16.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:16.822 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:16.838 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.837+0000 7f0fc5633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:16.840 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.839+0000 7f0fc5633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:16.841 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:16.840+0000 7f0fc5633780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:16.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:17.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:17.026 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:17.024+0000 7f0fc5633780 -1 Falling back to public interface 2026-02-13T19:34:17.380 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:17.379+0000 7f0fc5633780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:18.024 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:18.024 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:18.024 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:18.025 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:18.025 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:18.025 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:18.123 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/750124288,v1:127.0.0.1:6801/750124288] [v2:127.0.0.1:6802/750124288,v1:127.0.0.1:6803/750124288] exists,up bbeaefda-fd00-49e3-a19f-72e2ce0509b9 2026-02-13T19:34:18.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:18.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:18.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:18.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: test_flush_pg_stats: create_rbd_pool 2026-02-13T19:34:18.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:34:18.218 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:34:18.224 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:34:18.224 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:34:18.499 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:34:18.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:34:19.507 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:34:19.788 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: test_flush_pg_stats: rados -p rbd put obj /etc/group 2026-02-13T19:34:19.805 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: test_flush_pg_stats: flush_pg_stats 2026-02-13T19:34:19.805 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:34:19.805 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:34:20.032 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:34:20.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:34:20.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:34:20.033 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:34:20.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:34:20.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:34:20.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:34:20.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:34:20.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:20.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:34:20.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:34:20.101 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:20.101 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:34:20.101 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:34:20.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:34:20.102 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:34:20.102 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:20.333 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:34:20.333 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: test_flush_pg_stats: local 'jq_filter=.pools | .[] | select(.name == "rbd") | .stats' 2026-02-13T19:34:20.334 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: ceph df detail --format=json 2026-02-13T19:34:20.334 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored' 2026-02-13T19:34:20.649 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: stored=8139 2026-02-13T19:34:20.650 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: ceph df detail --format=json 2026-02-13T19:34:20.650 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored_raw' 2026-02-13T19:34:20.966 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: stored_raw=8139 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: test 8139 -gt 0 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2293: test_flush_pg_stats: test 8139 == 8139 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: test_flush_pg_stats: teardown td/ceph-helpers 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:20.967 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:21.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:21.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:21.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:21.075 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:21.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:21.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:21.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:21.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:21.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:21.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:21.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:21.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:21.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:21.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:21.084 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:21.084 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.084 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:21.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:21.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:21.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:21.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_config td/ceph-helpers 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: test_get_config: local dir=td/ceph-helpers 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: test_get_config: setup td/ceph-helpers 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:21.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:21.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:21.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:21.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:21.089 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:21.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:21.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:21.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:21.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:21.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:21.091 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:21.091 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:21.091 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:21.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:21.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:21.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:21.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.094 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:21.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:21.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:21.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:21.096 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:21.096 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.096 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.096 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:21.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:21.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:21.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:21.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:21.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1131: test_get_config: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:21.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:21.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:21.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:21.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:21.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:21.100 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:21.116 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:21.116 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:21.117 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:21.117 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:21.117 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.117 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.117 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:21.117 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:21.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.143 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:21.144 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:21.144 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:21.186 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.187 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.188 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:21.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:21.188 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:21.228 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: get_config mon a osd_pool_default_size 2026-02-13T19:34:21.228 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:21.228 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:21.228 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_pool_default_size 2026-02-13T19:34:21.229 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_pool_default_size 2026-02-13T19:34:21.229 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:21.229 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:21.229 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:21.230 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:21.230 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.230 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.230 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:21.230 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:21.230 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get osd_pool_default_size 2026-02-13T19:34:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: test 1 = 1 2026-02-13T19:34:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1133: test_get_config: run_mgr td/ceph-helpers x 2026-02-13T19:34:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:21.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:21.373 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:21.373 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:21.373 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:21.374 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:21.374 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.374 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.374 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:21.374 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:21.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:21.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: test_get_config: run_osd td/ceph-helpers 0 --osd_max_scrubs=3 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:21.393 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:21.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_max_scrubs=3 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:21.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:21.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cedac2fc-9336-4ad1-a199-7b69b22e4645 2026-02-13T19:34:21.396 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 cedac2fc-9336-4ad1-a199-7b69b22e4645 2026-02-13T19:34:21.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 cedac2fc-9336-4ad1-a199-7b69b22e4645' 2026-02-13T19:34:21.397 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:21.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC9fI9piAj6FxAAz5gDLIYkcQ/4fYn/kYt5KQ== 2026-02-13T19:34:21.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC9fI9piAj6FxAAz5gDLIYkcQ/4fYn/kYt5KQ=="}' 2026-02-13T19:34:21.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cedac2fc-9336-4ad1-a199-7b69b22e4645 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:21.504 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:21.510 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:21.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 --mkfs --key AQC9fI9piAj6FxAAz5gDLIYkcQ/4fYn/kYt5KQ== --osd-uuid cedac2fc-9336-4ad1-a199-7b69b22e4645 2026-02-13T19:34:21.528 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:21.526+0000 7fbebb644780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:21.529 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:21.528+0000 7fbebb644780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:21.530 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:21.529+0000 7fbebb644780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:21.530 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:21.529+0000 7fbebb644780 -1 bdev(0x2caac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:21.531 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:21.529+0000 7fbebb644780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:21.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:21.952 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:21.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:21.953 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:21.953 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:22.058 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:22.059 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:22.059 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 2026-02-13T19:34:22.059 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:22.059 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:22.059 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:22.073 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:22.071+0000 7f6d47832780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:22.074 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:22.073+0000 7f6d47832780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:22.075 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:22.074+0000 7f6d47832780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:22.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:22.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:22.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:22.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:22.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:22.159 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:22.159 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:22.159 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:22.159 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:22.160 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:22.250 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:22.248+0000 7f6d47832780 -1 Falling back to public interface 2026-02-13T19:34:22.257 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:22.601 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:22.599+0000 7f6d47832780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:23.258 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:23.259 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:23.259 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:23.259 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:23.260 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:23.260 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:23.356 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2220878329,v1:127.0.0.1:6801/2220878329] [v2:127.0.0.1:6802/2220878329,v1:127.0.0.1:6803/2220878329] exists,up cedac2fc-9336-4ad1-a199-7b69b22e4645 2026-02-13T19:34:23.358 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:23.358 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:23.358 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: get_config osd 0 osd_max_scrubs 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_max_scrubs 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_max_scrubs 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:34:23.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:23.360 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_max_scrubs 2026-02-13T19:34:23.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: test 3 = 3 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1136: test_get_config: teardown td/ceph-helpers 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:23.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:23.513 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:23.513 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:23.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:23.514 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:23.515 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:23.516 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:23.516 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:23.517 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:23.517 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:23.518 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:23.518 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:23.518 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:23.519 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:23.520 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:23.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:23.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.523 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:23.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_is_making_recovery_progress td/ceph-helpers 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: test_get_is_making_recovery_progress: local dir=td/ceph-helpers 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: test_get_is_making_recovery_progress: setup td/ceph-helpers 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:23.525 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:23.526 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:23.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:23.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:23.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:23.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:23.526 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:23.527 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:23.527 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:23.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:23.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:23.528 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:23.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:23.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:23.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:23.529 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:23.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:23.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:23.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:23.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:23.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:23.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:23.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:23.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:23.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:23.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:23.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:23.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:23.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:23.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:23.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1338: test_get_is_making_recovery_progress: run_mon td/ceph-helpers a 2026-02-13T19:34:23.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:23.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:23.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:23.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:23.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:23.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:34:23.555 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:23.555 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:23.555 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:23.555 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:23.555 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.555 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.555 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:23.556 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:34:23.580 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:23.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.582 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:23.583 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:23.583 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:23.625 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:23.625 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.626 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:23.627 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:23.627 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:23.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: test_get_is_making_recovery_progress: run_mgr td/ceph-helpers x 2026-02-13T19:34:23.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:23.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:23.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:23.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:23.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:23.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:23.773 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:23.773 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:23.773 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:23.773 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:23.774 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:23.774 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:23.774 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:23.774 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:23.774 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:23.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: test_get_is_making_recovery_progress: get_is_making_recovery_progress 2026-02-13T19:34:23.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T19:34:23.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T19:34:23.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T19:34:23.791 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T19:34:23.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T19:34:23.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T19:34:23.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T19:34:23.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T19:34:23.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1341: test_get_is_making_recovery_progress: teardown td/ceph-helpers 2026-02-13T19:34:23.891 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:23.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:23.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:23.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:23.892 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:23.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:23.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:23.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:23.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:23.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:23.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:23.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:24.000 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:24.000 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:24.001 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:24.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:24.001 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:24.002 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:24.002 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:24.003 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:24.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:24.004 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:24.005 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:24.006 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.006 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.006 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:24.007 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_last_scrub_stamp td/ceph-helpers 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1524: test_get_last_scrub_stamp: local dir=td/ceph-helpers 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: test_get_last_scrub_stamp: setup td/ceph-helpers 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:24.008 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:24.009 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:24.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:24.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:24.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:24.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:24.009 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:24.010 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:24.010 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:24.011 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:24.011 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:24.011 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:24.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:24.012 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:24.013 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:24.013 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:24.013 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:24.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:24.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:24.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:24.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:24.016 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:24.016 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:24.016 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:24.017 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:24.017 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.017 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.017 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:24.018 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:24.018 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: test_get_last_scrub_stamp: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:24.019 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:24.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:24.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:24.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:24.037 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:24.037 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.037 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.038 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:24.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:24.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:24.062 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:24.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:24.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:24.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:24.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:24.064 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:24.064 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:24.064 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:24.064 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:24.064 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.064 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.065 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:24.065 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:24.065 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:24.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:24.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:24.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:24.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:24.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:24.109 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:24.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: test_get_last_scrub_stamp: run_mgr td/ceph-helpers x 2026-02-13T19:34:24.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:24.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:24.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:24.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:24.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:24.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:24.253 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:24.253 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:24.254 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:24.254 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:24.254 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.254 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.254 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:24.255 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:24.255 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:24.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: test_get_last_scrub_stamp: run_osd td/ceph-helpers 0 2026-02-13T19:34:24.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:24.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:24.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:24.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:24.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:24.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:24.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:24.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:24.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:24.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:24.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:24.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:24.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:24.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:24.278 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:24.278 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:24.278 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:24.278 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:24.278 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:24.278 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:24.278 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:24.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:24.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:24.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:24.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:24.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:24.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:24.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:24.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:24.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:24.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:24.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:24.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:24.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:24.280 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:24.281 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5d1e5ac6-3cd4-4be0-b92d-352790a9c4e0 2026-02-13T19:34:24.281 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5d1e5ac6-3cd4-4be0-b92d-352790a9c4e0' 2026-02-13T19:34:24.281 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 5d1e5ac6-3cd4-4be0-b92d-352790a9c4e0 2026-02-13T19:34:24.281 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:24.283 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDAfI9paWrQEBAAwtkrutPa/+sqgdTBNbb8Jw== 2026-02-13T19:34:24.283 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDAfI9paWrQEBAAwtkrutPa/+sqgdTBNbb8Jw=="}' 2026-02-13T19:34:24.283 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5d1e5ac6-3cd4-4be0-b92d-352790a9c4e0 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:24.385 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:24.391 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:24.392 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDAfI9paWrQEBAAwtkrutPa/+sqgdTBNbb8Jw== --osd-uuid 5d1e5ac6-3cd4-4be0-b92d-352790a9c4e0 2026-02-13T19:34:24.408 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.406+0000 7fd91205e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:24.409 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.408+0000 7fd91205e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:24.410 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.409+0000 7fd91205e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:24.410 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.409+0000 7fd91205e780 -1 bdev(0x40c0c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:24.410 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.409+0000 7fd91205e780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:24.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:24.826 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:24.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:24.827 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:24.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:24.933 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:24.934 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:24.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:24.934 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:24.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:24.935 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:24.949 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.948+0000 7f97b1a5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:24.950 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.949+0000 7f97b1a5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:24.951 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:24.950+0000 7f97b1a5c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:25.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:25.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:25.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:25.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:25.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:25.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:25.033 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:25.034 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:25.034 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:25.034 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:25.123 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:25.122+0000 7f97b1a5c780 -1 Falling back to public interface 2026-02-13T19:34:25.129 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:25.507 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:25.506+0000 7f97b1a5c780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:26.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:26.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:26.130 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:26.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:26.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:26.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:26.231 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3354290229,v1:127.0.0.1:6801/3354290229] [v2:127.0.0.1:6802/3354290229,v1:127.0.0.1:6803/3354290229] exists,up 5d1e5ac6-3cd4-4be0-b92d-352790a9c4e0 2026-02-13T19:34:26.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:26.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:26.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:26.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1530: test_get_last_scrub_stamp: create_rbd_pool 2026-02-13T19:34:26.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:34:26.329 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:34:26.335 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:34:26.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:34:26.566 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:34:26.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:34:27.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:34:27.855 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1531: test_get_last_scrub_stamp: wait_for_clean 2026-02-13T19:34:27.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:34:27.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:34:27.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:34:27.856 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:34:27.857 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:34:27.857 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:34:27.857 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:34:27.857 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:34:27.857 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:34:27.938 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:34:27.938 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:34:27.938 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:34:27.938 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:34:27.938 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:34:27.938 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:34:28.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:34:28.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:34:28.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:34:28.165 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:34:28.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:34:28.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:34:28.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:34:28.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:34:28.232 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:28.232 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:34:28.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:34:28.233 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:28.233 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:34:28.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:34:28.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:34:28.234 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:34:28.235 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:28.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-13T19:34:28.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:34:29.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:34:29.463 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:29.695 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-13T19:34:29.695 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:34:30.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:34:30.696 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:30.926 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:34:30.927 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:34:30.927 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:30.927 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:31.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:34:31.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:34:31.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:34:31.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:34:31.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:34:31.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:34:31.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:34:31.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:34:31.470 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:34:31.470 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:34:31.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:31.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:31.788 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:34:31.788 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:34:31.789 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:34:31.789 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: get_last_scrub_stamp 1.0 2026-02-13T19:34:31.789 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:34:31.789 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:34:31.789 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:34:31.789 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:34:32.014 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: stamp=2026-02-13T19:34:26.501926+0000 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1533: test_get_last_scrub_stamp: test -n 2026-02-13T19:34:26.501926+0000 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1534: test_get_last_scrub_stamp: teardown td/ceph-helpers 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:32.015 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:32.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:32.121 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:32.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:32.122 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:32.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:32.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:32.124 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:32.124 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:32.125 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:32.125 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:32.125 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:32.126 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:32.127 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:32.127 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:32.130 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:32.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.131 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:32.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:32.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:32.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:32.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_not_primary td/ceph-helpers 2026-02-13T19:34:32.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: test_get_not_primary: local dir=td/ceph-helpers 2026-02-13T19:34:32.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1231: test_get_not_primary: setup td/ceph-helpers 2026-02-13T19:34:32.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:32.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:32.135 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:32.135 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:32.135 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:32.135 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:32.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:32.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:32.136 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:32.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:32.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:32.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:32.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:32.138 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:32.139 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:32.139 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:32.140 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:32.140 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.140 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:32.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:32.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:32.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:32.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:32.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.143 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:32.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: test_get_not_primary: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:32.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:32.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-13T19:34:32.188 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:32.190 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:32.191 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:32.191 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.191 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.191 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:32.192 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:32.192 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:32.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:32.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:32.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:32.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:32.234 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:32.235 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:32.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: test_get_not_primary: run_mgr td/ceph-helpers x 2026-02-13T19:34:32.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:32.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:32.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:32.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:32.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:32.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:32.378 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:32.378 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:32.378 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:32.379 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:32.379 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.379 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:32.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:32.380 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:32.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: test_get_not_primary: run_osd td/ceph-helpers 0 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:32.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:32.398 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:32.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:32.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:32.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:32.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:32.400 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:32.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=53546144-53b4-4384-a7f9-6909524684bd 2026-02-13T19:34:32.402 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 53546144-53b4-4384-a7f9-6909524684bd 2026-02-13T19:34:32.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 53546144-53b4-4384-a7f9-6909524684bd' 2026-02-13T19:34:32.402 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:32.409 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDIfI9pWkpZGBAAhYx2qpxMNVBaqx4UO+PqiA== 2026-02-13T19:34:32.409 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDIfI9pWkpZGBAAhYx2qpxMNVBaqx4UO+PqiA=="}' 2026-02-13T19:34:32.409 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 53546144-53b4-4384-a7f9-6909524684bd -i td/ceph-helpers/0/new.json 2026-02-13T19:34:32.514 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:32.521 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:32.522 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDIfI9pWkpZGBAAhYx2qpxMNVBaqx4UO+PqiA== --osd-uuid 53546144-53b4-4384-a7f9-6909524684bd 2026-02-13T19:34:32.538 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:32.537+0000 7ff0cc00d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:32.539 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:32.538+0000 7ff0cc00d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:32.540 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:32.539+0000 7ff0cc00d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:32.541 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:32.540+0000 7ff0cc00d780 -1 bdev(0x2e88c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:32.541 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:32.540+0000 7ff0cc00d780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:32.958 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:32.958 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:32.959 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:32.959 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:32.959 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:33.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:33.064 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:33.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:33.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:33.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:33.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:33.078 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:33.077+0000 7fa750e12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:33.079 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:33.078+0000 7fa750e12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:33.080 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:33.079+0000 7fa750e12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:33.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:33.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:33.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:33.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:33.165 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:33.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:33.166 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:33.166 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:33.167 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:33.167 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:33.264 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:33.279 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:33.278+0000 7fa750e12780 -1 Falling back to public interface 2026-02-13T19:34:33.712 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:33.710+0000 7fa750e12780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:34.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:34.265 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:34.266 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:34.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:34.266 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:34.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:34.364 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1138876020,v1:127.0.0.1:6801/1138876020] [v2:127.0.0.1:6802/1138876020,v1:127.0.0.1:6803/1138876020] exists,up 53546144-53b4-4384-a7f9-6909524684bd 2026-02-13T19:34:34.368 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:34.368 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:34.368 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:34.368 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1235: test_get_not_primary: run_osd td/ceph-helpers 1 2026-02-13T19:34:34.368 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T19:34:34.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:34.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:34.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e61dcd3-51ca-46f5-852d-639ca79de633 2026-02-13T19:34:34.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4e61dcd3-51ca-46f5-852d-639ca79de633' 2026-02-13T19:34:34.372 INFO:tasks.workunit.client.0.trial162.stdout:add osd1 4e61dcd3-51ca-46f5-852d-639ca79de633 2026-02-13T19:34:34.372 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:34.377 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDKfI9pIBJ6FhAALvQ8d0+rzen5YD3QaBRq8Q== 2026-02-13T19:34:34.378 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDKfI9pIBJ6FhAALvQ8d0+rzen5YD3QaBRq8Q=="}' 2026-02-13T19:34:34.378 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e61dcd3-51ca-46f5-852d-639ca79de633 -i td/ceph-helpers/1/new.json 2026-02-13T19:34:34.482 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:34.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T19:34:34.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDKfI9pIBJ6FhAALvQ8d0+rzen5YD3QaBRq8Q== --osd-uuid 4e61dcd3-51ca-46f5-852d-639ca79de633 2026-02-13T19:34:34.504 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:34.502+0000 7f7df7833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:34.505 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:34.504+0000 7f7df7833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:34.506 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:34.504+0000 7f7df7833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:34.506 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:34.505+0000 7f7df7833780 -1 bdev(0x3d4ac00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:34.507 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:34.505+0000 7f7df7833780 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T19:34:34.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T19:34:34.824 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:34.824 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:34:34.825 INFO:tasks.workunit.client.0.trial162.stdout:adding osd1 key to auth repository 2026-02-13T19:34:34.825 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:35.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:34:35.149 INFO:tasks.workunit.client.0.trial162.stdout:start osd.1 2026-02-13T19:34:35.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:35.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:35.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:35.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:35.162 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:35.161+0000 7fbfcc3e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:35.163 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:35.162+0000 7fbfcc3e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:35.164 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:35.163+0000 7fbfcc3e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:35.300 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:35.298+0000 7fbfcc3e3780 -1 Falling back to public interface 2026-02-13T19:34:35.376 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:34:35.376 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:35.376 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:34:35.376 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:35.377 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:35.377 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:35.377 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:35.377 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:35.378 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:35.378 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:34:35.601 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:35.599+0000 7fbfcc3e3780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:34:35.606 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:36.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:36.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:36.608 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:36.609 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:36.609 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:36.609 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:34:36.835 INFO:tasks.workunit.client.0.trial162.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1686654798,v1:127.0.0.1:6811/1686654798] [v2:127.0.0.1:6812/1686654798,v1:127.0.0.1:6813/1686654798] exists,up 4e61dcd3-51ca-46f5-852d-639ca79de633 2026-02-13T19:34:36.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:36.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:36.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:36.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1236: test_get_not_primary: create_rbd_pool 2026-02-13T19:34:36.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:34:37.058 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:34:37.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:34:37.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:34:37.312 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:34:37.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:34:38.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:34:38.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1237: test_get_not_primary: wait_for_clean 2026-02-13T19:34:38.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:34:38.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:34:38.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:34:38.602 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:34:38.602 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:34:38.603 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:34:38.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:34:38.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:34:38.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:34:38.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:34:38.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:34:38.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:34:38.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:34:38.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:34:38.684 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:34:38.910 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:34:38.911 INFO:tasks.workunit.client.0.trial162.stderr:1' 2026-02-13T19:34:38.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:34:38.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:34:38.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:34:38.975 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:34:38.975 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:34:38.976 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:34:38.976 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:34:38.976 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:34:39.043 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-13T19:34:39.043 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-13T19:34:39.043 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483 1-42949672963' 2026-02-13T19:34:39.043 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:34:39.043 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:39.043 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:34:39.044 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:34:39.044 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:39.044 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:34:39.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:34:39.046 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:34:39.046 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:34:39.046 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:39.272 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:34:39.272 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:34:40.272 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:34:40.272 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:40.501 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-13T19:34:40.501 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:34:40.501 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672963 2026-02-13T19:34:40.501 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:34:40.501 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:34:40.502 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-13T19:34:40.502 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:34:40.503 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672963 2026-02-13T19:34:40.504 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-13T19:34:40.504 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 42949672963 2026-02-13T19:34:40.504 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:34:40.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672963 -lt 42949672963 2026-02-13T19:34:40.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:34:40.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:40.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:41.075 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:34:41.075 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:34:41.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:34:41.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:34:41.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:34:41.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:34:41.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:34:41.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:34:41.298 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:34:41.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:34:41.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:41.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:41.620 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:34:41.620 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:34:41.621 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:34:41.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: get_primary rbd GROUP 2026-02-13T19:34:41.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-13T19:34:41.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-13T19:34:41.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-13T19:34:41.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T19:34:41.851 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: local primary=1 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: get_not_primary rbd GROUP 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=GROUP 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd GROUP 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-13T19:34:41.852 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T19:34:42.083 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-13T19:34:42.083 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd GROUP 2026-02-13T19:34:42.083 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-13T19:34:42.327 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: local not_primary=0 2026-02-13T19:34:42.327 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1240: test_get_not_primary: test 0 '!=' 1 2026-02-13T19:34:42.327 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1241: test_get_not_primary: test 0 = 0 -o 0 = 1 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1242: test_get_not_primary: teardown td/ceph-helpers 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:42.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:42.436 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:42.436 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:42.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:42.438 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:42.439 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:42.439 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:42.439 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:42.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:42.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:42.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:42.441 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:42.441 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:42.442 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:42.442 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:42.447 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:42.447 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.447 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:42.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:42.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:42.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:42.448 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_clean td/ceph-helpers 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: test_get_num_active_clean: local dir=td/ceph-helpers 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: test_get_num_active_clean: setup td/ceph-helpers 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:42.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:42.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:42.451 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:42.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:42.452 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:42.453 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:42.453 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:42.453 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:42.454 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:42.454 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:42.454 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:42.455 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:42.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:42.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:42.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:42.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:42.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.457 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.457 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:42.458 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:42.458 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:42.458 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:42.460 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:42.460 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.460 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:42.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:42.461 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: test_get_num_active_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:42.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:42.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:42.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:42.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:42.481 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:42.481 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.481 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.481 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:42.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:42.507 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:42.507 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:42.507 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:42.507 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:42.507 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:42.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:42.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:42.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:42.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:42.550 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:42.551 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:42.551 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:42.552 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:42.592 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: test_get_num_active_clean: run_mgr td/ceph-helpers x 2026-02-13T19:34:42.592 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:42.592 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:42.593 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:42.593 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:42.593 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:42.593 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:42.694 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:42.694 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:42.694 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:42.694 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:42.694 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.694 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.694 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:42.695 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:42.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: test_get_num_active_clean: run_osd td/ceph-helpers 0 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:42.713 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:42.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:42.715 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:42.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dc1eca0e-ef3e-4d45-a803-5ba9c720ceb4 2026-02-13T19:34:42.715 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dc1eca0e-ef3e-4d45-a803-5ba9c720ceb4' 2026-02-13T19:34:42.715 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 dc1eca0e-ef3e-4d45-a803-5ba9c720ceb4 2026-02-13T19:34:42.716 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:42.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDSfI9pEqU5KxAAZ18DkFixrxoon8OLT4n58g== 2026-02-13T19:34:42.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDSfI9pEqU5KxAAZ18DkFixrxoon8OLT4n58g=="}' 2026-02-13T19:34:42.725 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dc1eca0e-ef3e-4d45-a803-5ba9c720ceb4 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:42.829 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:42.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:42.836 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDSfI9pEqU5KxAAZ18DkFixrxoon8OLT4n58g== --osd-uuid dc1eca0e-ef3e-4d45-a803-5ba9c720ceb4 2026-02-13T19:34:42.852 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:42.851+0000 7f101545e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:42.853 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:42.852+0000 7f101545e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:42.854 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:42.853+0000 7f101545e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:42.855 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:42.854+0000 7f101545e780 -1 bdev(0x2cb2c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:42.855 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:42.854+0000 7f101545e780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:43.245 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:43.245 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:43.246 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:43.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:43.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:43.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:43.352 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:43.352 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:43.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:43.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:43.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:43.366 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:43.365+0000 7fda9abd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:43.368 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:43.367+0000 7fda9abd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:43.369 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:43.368+0000 7fda9abd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:43.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:43.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:43.449 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:43.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:43.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:43.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:43.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:43.450 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:43.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:43.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:43.546 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:43.566 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:43.565+0000 7fda9abd6780 -1 Falling back to public interface 2026-02-13T19:34:43.905 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:43.903+0000 7fda9abd6780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:44.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:44.547 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:44.548 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:44.548 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:44.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:44.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:44.644 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1535474208,v1:127.0.0.1:6801/1535474208] [v2:127.0.0.1:6802/1535474208,v1:127.0.0.1:6803/1535474208] exists,up dc1eca0e-ef3e-4d45-a803-5ba9c720ceb4 2026-02-13T19:34:44.644 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:44.644 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:44.644 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:44.644 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1369: test_get_num_active_clean: create_rbd_pool 2026-02-13T19:34:44.644 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:34:44.745 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:34:44.751 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:34:44.751 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:34:45.067 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:34:45.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:34:46.075 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:34:46.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1370: test_get_num_active_clean: wait_for_clean 2026-02-13T19:34:46.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:34:46.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:34:46.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:34:46.354 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:34:46.354 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:34:46.354 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:34:46.354 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:34:46.355 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:34:46.355 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:34:46.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:34:46.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:34:46.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:34:46.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:34:46.437 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:34:46.437 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:34:46.665 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:34:46.666 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:34:46.666 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:34:46.666 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:34:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:34:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:34:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:34:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:34:46.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:46.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:34:46.733 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:34:46.733 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:46.733 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:34:46.734 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:34:46.734 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:34:46.734 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:34:46.735 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:46.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:34:46.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:34:47.963 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:34:47.963 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:48.202 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:34:48.202 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:34:49.203 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:34:49.203 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:49.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:34:49.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:34:49.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:49.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:49.750 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:34:49.750 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:34:49.750 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:34:49.750 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:34:49.750 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:34:49.750 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:34:49.750 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:34:49.751 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:34:49.972 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:34:49.973 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:34:49.973 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:49.973 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:50.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:34:50.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:34:50.299 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:34:50.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: get_num_active_clean 2026-02-13T19:34:50.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:34:50.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:34:50.300 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:34:50.300 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:34:50.300 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: local num_active_clean=4 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1372: test_get_num_active_clean: test 4 = 4 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1373: test_get_num_active_clean: teardown td/ceph-helpers 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:50.523 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:50.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:50.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:50.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:50.629 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:50.630 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:50.630 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:50.631 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:50.632 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:50.632 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:50.632 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:50.633 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:50.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:50.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:50.634 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:50.635 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:50.636 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:50.636 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:50.639 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:50.639 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.639 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.640 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_or_peered td/ceph-helpers 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1393: test_get_num_active_or_peered: local dir=td/ceph-helpers 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1395: test_get_num_active_or_peered: setup td/ceph-helpers 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:50.642 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:50.643 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:50.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:50.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:50.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:50.643 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:50.643 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:50.644 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:50.644 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:50.645 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:50.645 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:50.645 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:50.646 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:50.646 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:50.646 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:50.646 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:50.647 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:50.648 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:50.648 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:50.649 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:50.649 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.649 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.649 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:50.650 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:50.650 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:50.650 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:50.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:50.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.651 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.651 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:50.652 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:50.653 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1396: test_get_num_active_or_peered: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:50.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:50.673 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:50.673 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:50.673 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:50.673 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:50.673 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.673 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.673 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:50.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:50.698 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:50.699 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.700 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.700 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:50.700 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:50.700 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:50.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:50.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:50.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:50.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:50.741 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:50.742 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:50.742 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:50.742 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:50.743 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:50.743 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.743 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.743 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:50.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:50.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:50.783 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1397: test_get_num_active_or_peered: run_mgr td/ceph-helpers x 2026-02-13T19:34:50.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:50.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:50.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:50.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:50.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:50.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:50.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:50.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:50.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:50.885 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:50.885 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.885 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:50.886 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:50.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:50.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1398: test_get_num_active_or_peered: run_osd td/ceph-helpers 0 2026-02-13T19:34:50.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:50.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:50.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:50.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:50.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:50.905 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:50.906 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:50.907 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:50.907 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:50.907 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:50.907 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:50.907 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:50.907 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8b562bc3-b3e9-437a-9e9b-a30300d58af4 2026-02-13T19:34:50.908 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8b562bc3-b3e9-437a-9e9b-a30300d58af4' 2026-02-13T19:34:50.908 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 8b562bc3-b3e9-437a-9e9b-a30300d58af4 2026-02-13T19:34:50.908 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:50.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDafI9ptrWCNhAAAWDnd2zaABk7LnPzTxgoXw== 2026-02-13T19:34:50.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDafI9ptrWCNhAAAWDnd2zaABk7LnPzTxgoXw=="}' 2026-02-13T19:34:50.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8b562bc3-b3e9-437a-9e9b-a30300d58af4 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:51.020 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:51.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:51.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDafI9ptrWCNhAAAWDnd2zaABk7LnPzTxgoXw== --osd-uuid 8b562bc3-b3e9-437a-9e9b-a30300d58af4 2026-02-13T19:34:51.044 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.044+0000 7fddf5659780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:51.045 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.045+0000 7fddf5659780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:51.046 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.046+0000 7fddf5659780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:51.047 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.046+0000 7fddf5659780 -1 bdev(0x35b8c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:51.047 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.046+0000 7fddf5659780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:51.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:51.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:51.480 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:51.480 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:51.480 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:51.584 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:51.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:51.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:51.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:51.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:51.586 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:51.600 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.600+0000 7fbaa1a2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:51.601 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.601+0000 7fbaa1a2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:51.602 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.602+0000 7fbaa1a2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:51.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:51.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:51.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:51.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:51.686 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:51.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:51.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:51.687 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:51.687 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:51.688 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:51.783 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:34:51.815 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:51.814+0000 7fbaa1a2d780 -1 Falling back to public interface 2026-02-13T19:34:52.134 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:52.134+0000 7fbaa1a2d780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:34:52.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:34:52.784 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:52.785 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:34:52.785 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:34:52.785 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:52.785 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:52.883 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/69856033,v1:127.0.0.1:6801/69856033] [v2:127.0.0.1:6802/69856033,v1:127.0.0.1:6803/69856033] exists,up 8b562bc3-b3e9-437a-9e9b-a30300d58af4 2026-02-13T19:34:52.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:34:52.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:34:52.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:34:52.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1399: test_get_num_active_or_peered: create_rbd_pool 2026-02-13T19:34:52.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:34:52.981 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:34:52.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:34:52.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:34:53.298 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:34:53.305 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:34:54.306 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:34:54.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1400: test_get_num_active_or_peered: wait_for_clean 2026-02-13T19:34:54.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:34:54.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:34:54.587 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:34:54.587 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:34:54.587 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:34:54.587 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:34:54.588 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:34:54.588 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:34:54.588 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:34:54.668 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:34:54.668 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:34:54.668 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:34:54.668 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:34:54.668 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:34:54.668 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:34:54.903 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:34:54.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:34:54.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:34:54.904 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:34:54.968 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:34:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:34:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:34:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:34:54.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:54.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:34:54.970 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:34:54.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:34:54.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:34:54.971 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:34:54.972 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:34:54.972 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:34:54.972 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:55.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:34:55.202 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:34:56.202 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:34:56.203 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:56.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:34:56.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:34:57.434 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:34:57.435 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:34:57.663 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:34:57.664 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:34:57.664 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:57.664 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:57.986 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:34:57.986 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:34:57.987 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:34:57.987 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:34:57.987 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:34:57.987 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:34:57.987 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:34:57.987 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:34:58.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:34:58.212 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:34:58.212 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:34:58.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:34:58.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:34:58.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:34:58.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:34:58.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: get_num_active_or_peered 2026-02-13T19:34:58.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-13T19:34:58.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-13T19:34:58.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-13T19:34:58.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-13T19:34:58.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-13T19:34:58.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: local num_peered=4 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1402: test_get_num_active_or_peered: test 4 = 4 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1403: test_get_num_active_or_peered: teardown td/ceph-helpers 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:58.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:58.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:58.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:58.868 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:58.869 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:58.869 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:58.870 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:58.870 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:58.871 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:58.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:58.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:58.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:58.873 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:58.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:58.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:58.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:58.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:58.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:58.878 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:58.878 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_pgs td/ceph-helpers 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1420: test_get_num_pgs: local dir=td/ceph-helpers 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1422: test_get_num_pgs: setup td/ceph-helpers 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:34:58.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:34:58.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:34:58.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:34:58.882 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:34:58.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:34:58.882 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:34:58.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:34:58.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:34:58.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:34:58.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:58.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:34:58.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:34:58.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:34:58.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:34:58.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:34:58.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:34:58.887 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:34:58.887 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:58.888 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:58.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:34:58.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:34:58.888 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:34:58.889 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:34:58.889 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:34:58.890 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:58.890 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:58.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:34:58.891 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:34:58.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:34:58.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:34:58.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:34:58.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1423: test_get_num_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:58.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:34:58.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:34:58.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:34:58.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:34:58.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:34:58.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:58.911 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:34:58.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:34:58.937 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:58.938 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:58.939 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:58.939 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:34:58.983 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:34:58.984 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:34:58.984 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:58.984 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:58.984 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:34:58.984 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:34:58.984 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:34:59.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1424: test_get_num_pgs: run_mgr td/ceph-helpers x 2026-02-13T19:34:59.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:34:59.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:34:59.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:34:59.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:34:59.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:34:59.028 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:34:59.127 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:34:59.127 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:59.127 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:59.127 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:59.128 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:59.128 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:59.128 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:59.128 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:59.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:34:59.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: test_get_num_pgs: run_osd td/ceph-helpers 0 2026-02-13T19:34:59.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:34:59.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:34:59.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:34:59.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:34:59.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0d08dd94-5913-4d0a-b5b7-2abf9d686d39 2026-02-13T19:34:59.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0d08dd94-5913-4d0a-b5b7-2abf9d686d39' 2026-02-13T19:34:59.151 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 0d08dd94-5913-4d0a-b5b7-2abf9d686d39 2026-02-13T19:34:59.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:34:59.157 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDjfI9pUy9dCRAAyOQYdE/RtlH8ob+US4RaCQ== 2026-02-13T19:34:59.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDjfI9pUy9dCRAAyOQYdE/RtlH8ob+US4RaCQ=="}' 2026-02-13T19:34:59.158 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0d08dd94-5913-4d0a-b5b7-2abf9d686d39 -i td/ceph-helpers/0/new.json 2026-02-13T19:34:59.261 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:59.267 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:34:59.268 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDjfI9pUy9dCRAAyOQYdE/RtlH8ob+US4RaCQ== --osd-uuid 0d08dd94-5913-4d0a-b5b7-2abf9d686d39 2026-02-13T19:34:59.285 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.284+0000 7f4f11630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:59.286 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.286+0000 7f4f11630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:59.287 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.287+0000 7f4f11630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:59.287 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.287+0000 7f4f11630780 -1 bdev(0x3e0cc00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:34:59.287 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.287+0000 7f4f11630780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:34:59.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:34:59.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:34:59.684 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:34:59.685 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:34:59.685 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:34:59.787 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:34:59.788 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:34:59.789 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:34:59.789 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:34:59.789 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:34:59.789 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:34:59.802 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.802+0000 7f2247c27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:59.803 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.803+0000 7f2247c27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:59.804 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:34:59.804+0000 7f2247c27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:34:59.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:34:59.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:34:59.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:34:59.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:34:59.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:34:59.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:34:59.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:34:59.886 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:34:59.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:34:59.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:34:59.981 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:00.009 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:00.009+0000 7f2247c27780 -1 Falling back to public interface 2026-02-13T19:35:00.431 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:00.430+0000 7f2247c27780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:00.982 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:00.982 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:00.982 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:00.983 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:00.983 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:00.983 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:01.082 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1977492962,v1:127.0.0.1:6801/1977492962] [v2:127.0.0.1:6802/1977492962,v1:127.0.0.1:6803/1977492962] exists,up 0d08dd94-5913-4d0a-b5b7-2abf9d686d39 2026-02-13T19:35:01.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:01.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:01.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:01.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1426: test_get_num_pgs: create_rbd_pool 2026-02-13T19:35:01.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:35:01.176 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:35:01.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:35:01.182 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:35:01.462 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:35:01.470 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:35:02.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:35:02.751 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1427: test_get_num_pgs: wait_for_clean 2026-02-13T19:35:02.751 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:35:02.752 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:35:02.752 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:35:02.752 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:35:02.753 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:02.753 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:02.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:02.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:02.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:02.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:35:02.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:35:02.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:35:02.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:35:02.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:35:02.836 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:35:03.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:35:03.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:35:03.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:35:03.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:35:03.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:35:03.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:35:03.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:35:03.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:35:03.130 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:03.130 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:35:03.131 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:35:03.132 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:03.132 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:35:03.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:35:03.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:35:03.133 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:35:03.133 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:03.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:35:03.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:04.363 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:35:04.364 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:04.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:35:04.595 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:05.596 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:35:05.596 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:05.827 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:35:05.827 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:35:05.828 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:05.828 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:06.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:35:06.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:35:06.147 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:35:06.147 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:35:06.147 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:35:06.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:35:06.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:35:06.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:35:06.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:35:06.370 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:35:06.371 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:06.371 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:06.690 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:35:06.690 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:35:06.690 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:35:06.691 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: get_num_pgs 2026-02-13T19:35:06.691 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:06.691 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:07.021 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: local num_pgs=4 2026-02-13T19:35:07.021 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1429: test_get_num_pgs: test 4 -gt 0 2026-02-13T19:35:07.021 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1430: test_get_num_pgs: teardown td/ceph-helpers 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:07.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:07.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:07.129 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:07.129 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:07.130 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:07.131 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:07.131 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:07.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:07.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:07.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:07.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:07.133 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:07.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:07.135 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:07.135 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:07.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:07.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.139 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:07.140 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:07.140 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_op_scheduler td/ceph-helpers 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: test_get_op_scheduler: local dir=td/ceph-helpers 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: test_get_op_scheduler: setup td/ceph-helpers 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:07.141 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:07.142 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:07.142 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:07.142 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:07.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:07.143 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:07.143 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:07.144 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:07.144 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:07.144 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:07.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:07.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:07.145 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:07.146 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:07.146 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:07.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:07.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:07.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:07.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:07.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:07.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:07.149 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:35:07.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:35:07.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:35:07.152 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:35:07.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:35:07.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:35:07.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:35:07.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: test_get_op_scheduler: run_mon td/ceph-helpers a 2026-02-13T19:35:07.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:35:07.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:35:07.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:35:07.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:35:07.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:35:07.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:07.171 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:35:07.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:35:07.197 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:35:07.197 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:07.197 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:07.197 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:35:07.197 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:35:07.198 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:07.198 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:07.198 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:07.199 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:07.199 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.199 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.199 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:07.199 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:07.199 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:07.239 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:07.240 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:07.240 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.240 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.240 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:07.241 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:07.241 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:35:07.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: test_get_op_scheduler: run_mgr td/ceph-helpers x 2026-02-13T19:35:07.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:35:07.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:35:07.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:35:07.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:35:07.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:35:07.282 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:35:07.384 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:35:07.384 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:07.384 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:07.384 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:07.385 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.385 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.385 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:07.385 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:07.385 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:07.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2322: test_get_op_scheduler: run_osd td/ceph-helpers 0 --osd_op_queue=wpq 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:35:07.404 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:07.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=wpq 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:07.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8efaecc0-95c4-4009-849f-ffc54d7740f0 2026-02-13T19:35:07.407 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 8efaecc0-95c4-4009-849f-ffc54d7740f0 2026-02-13T19:35:07.407 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8efaecc0-95c4-4009-849f-ffc54d7740f0' 2026-02-13T19:35:07.407 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:07.414 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDrfI9pay2oGBAA5fB8u/mrpyOsmgs77BNE6Q== 2026-02-13T19:35:07.414 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDrfI9pay2oGBAA5fB8u/mrpyOsmgs77BNE6Q=="}' 2026-02-13T19:35:07.414 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8efaecc0-95c4-4009-849f-ffc54d7740f0 -i td/ceph-helpers/0/new.json 2026-02-13T19:35:07.520 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:07.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:35:07.527 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq --mkfs --key AQDrfI9pay2oGBAA5fB8u/mrpyOsmgs77BNE6Q== --osd-uuid 8efaecc0-95c4-4009-849f-ffc54d7740f0 2026-02-13T19:35:07.543 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:07.542+0000 7fcb4242c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:07.545 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:07.544+0000 7fcb4242c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:07.546 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:07.545+0000 7fcb4242c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:07.547 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:07.545+0000 7fcb4242c780 -1 bdev(0x3380c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:07.547 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:07.545+0000 7fcb4242c780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:35:07.976 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:35:07.976 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:07.977 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:35:07.977 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:35:07.977 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:08.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:35:08.081 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:35:08.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq 2026-02-13T19:35:08.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:08.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:08.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:08.096 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:08.095+0000 7f5506243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:08.097 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:08.096+0000 7f5506243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:08.098 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:08.097+0000 7f5506243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:08.178 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:35:08.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:08.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:35:08.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:08.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:08.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:08.179 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:08.180 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:08.180 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:08.180 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:08.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:08.287 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:08.286+0000 7f5506243780 -1 Falling back to public interface 2026-02-13T19:35:08.664 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:08.663+0000 7f5506243780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:09.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:09.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:09.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:09.279 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:09.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:09.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:09.377 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3658362195,v1:127.0.0.1:6801/3658362195] [v2:127.0.0.1:6802/3658362195,v1:127.0.0.1:6803/3658362195] exists,up 8efaecc0-95c4-4009-849f-ffc54d7740f0 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: get_op_scheduler 0 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=0 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 0 osd_op_queue 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-13T19:35:09.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.0.asok 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:09.380 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-osd.0.asok config get osd_op_queue 2026-02-13T19:35:09.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: test wpq = wpq 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2325: test_get_op_scheduler: run_osd td/ceph-helpers 1 --osd_op_queue=mclock_scheduler 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:09.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:09.432 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T19:35:09.433 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:09.434 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=77d03d0c-791a-4cdb-8abf-54aa5a351503 2026-02-13T19:35:09.434 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 77d03d0c-791a-4cdb-8abf-54aa5a351503' 2026-02-13T19:35:09.434 INFO:tasks.workunit.client.0.trial162.stdout:add osd1 77d03d0c-791a-4cdb-8abf-54aa5a351503 2026-02-13T19:35:09.434 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:09.441 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDtfI9pAE1AGhAA4TbFwQHqYG3VbUkel1P8VQ== 2026-02-13T19:35:09.441 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDtfI9pAE1AGhAA4TbFwQHqYG3VbUkel1P8VQ=="}' 2026-02-13T19:35:09.441 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 77d03d0c-791a-4cdb-8abf-54aa5a351503 -i td/ceph-helpers/1/new.json 2026-02-13T19:35:09.546 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:09.552 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T19:35:09.553 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQDtfI9pAE1AGhAA4TbFwQHqYG3VbUkel1P8VQ== --osd-uuid 77d03d0c-791a-4cdb-8abf-54aa5a351503 2026-02-13T19:35:09.570 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:09.568+0000 7f6e7020f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:09.571 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:09.570+0000 7f6e7020f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:09.572 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:09.571+0000 7f6e7020f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:09.572 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:09.571+0000 7f6e7020f780 -1 bdev(0x3542c00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:09.572 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:09.571+0000 7f6e7020f780 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T19:35:09.899 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T19:35:09.899 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:09.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:35:09.900 INFO:tasks.workunit.client.0.trial162.stdout:adding osd1 key to auth repository 2026-02-13T19:35:09.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:10.220 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:35:10.220 INFO:tasks.workunit.client.0.trial162.stdout:start osd.1 2026-02-13T19:35:10.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-13T19:35:10.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:10.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:10.222 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:10.235 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:10.234+0000 7f3cd1431780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:10.237 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:10.236+0000 7f3cd1431780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:10.237 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:10.236+0000 7f3cd1431780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:10.371 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:10.370+0000 7f3cd1431780 -1 Falling back to public interface 2026-02-13T19:35:10.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:35:10.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:10.450 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:35:10.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:10.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:10.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:10.451 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:10.451 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:10.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:10.452 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:35:10.679 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:10.680 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:10.680+0000 7f3cd1431780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:35:11.680 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:11.681 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:11.681 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:11.682 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:11.682 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:11.682 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:35:11.911 INFO:tasks.workunit.client.0.trial162.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/612898088,v1:127.0.0.1:6811/612898088] [v2:127.0.0.1:6812/612898088,v1:127.0.0.1:6813/612898088] exists,up 77d03d0c-791a-4cdb-8abf-54aa5a351503 2026-02-13T19:35:11.913 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: get_op_scheduler 1 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=1 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 1 osd_op_queue 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=1 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.1 2026-02-13T19:35:11.914 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:35:11.915 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:35:11.915 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:11.915 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:11.915 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:11.915 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-osd.1.asok 2026-02-13T19:35:11.915 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:11.915 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-osd.1.asok config get osd_op_queue 2026-02-13T19:35:11.961 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: test mclock_scheduler = mclock_scheduler 2026-02-13T19:35:11.961 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: test_get_op_scheduler: teardown td/ceph-helpers 2026-02-13T19:35:11.961 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:11.961 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:11.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:11.962 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:11.962 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:11.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:11.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:11.962 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:12.068 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:12.068 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:12.069 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:12.069 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:12.070 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:12.071 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:12.071 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:12.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:12.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:12.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:12.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:12.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:12.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:12.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:12.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:12.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:12.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osd_id_used_by_pgs td/ceph-helpers 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1450: test_get_osd_id_used_by_pgs: local dir=td/ceph-helpers 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1452: test_get_osd_id_used_by_pgs: setup td/ceph-helpers 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:12.081 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:12.082 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:12.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:12.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:12.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:12.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:12.082 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:12.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:12.083 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:12.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:12.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:12.084 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:12.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:12.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:12.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:12.086 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:12.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:12.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:12.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:12.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:12.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.088 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:12.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:12.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:12.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:35:12.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:35:12.090 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.091 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.091 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:35:12.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1453: test_get_osd_id_used_by_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:35:12.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:12.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:35:12.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:12.112 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:12.112 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:12.113 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.113 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.113 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:12.113 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:12.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:35:12.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:35:12.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:12.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:12.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:35:12.139 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:35:12.140 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:12.140 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:12.140 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:12.140 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:12.140 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.140 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.140 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:12.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:12.141 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:35:12.182 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:35:12.182 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:12.182 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:12.182 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:35:12.182 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:12.184 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:35:12.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1454: test_get_osd_id_used_by_pgs: run_mgr td/ceph-helpers x 2026-02-13T19:35:12.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:35:12.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:35:12.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:35:12.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:35:12.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:35:12.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:35:12.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:35:12.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:12.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:12.329 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:12.329 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.329 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:12.330 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:12.330 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:12.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1455: test_get_osd_id_used_by_pgs: run_osd td/ceph-helpers 0 2026-02-13T19:35:12.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:12.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:12.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:12.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:12.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f07a7981-35e9-41b0-bd30-2ae773b29f07 2026-02-13T19:35:12.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f07a7981-35e9-41b0-bd30-2ae773b29f07' 2026-02-13T19:35:12.352 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 f07a7981-35e9-41b0-bd30-2ae773b29f07 2026-02-13T19:35:12.352 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:12.358 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDwfI9pVPZSFRAAL5FDH29/Ze1ynBbVPH/Spw== 2026-02-13T19:35:12.358 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDwfI9pVPZSFRAAL5FDH29/Ze1ynBbVPH/Spw=="}' 2026-02-13T19:35:12.358 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f07a7981-35e9-41b0-bd30-2ae773b29f07 -i td/ceph-helpers/0/new.json 2026-02-13T19:35:12.461 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:12.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:35:12.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDwfI9pVPZSFRAAL5FDH29/Ze1ynBbVPH/Spw== --osd-uuid f07a7981-35e9-41b0-bd30-2ae773b29f07 2026-02-13T19:35:12.485 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:12.483+0000 7ff3c4029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:12.486 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:12.485+0000 7ff3c4029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:12.487 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:12.486+0000 7ff3c4029780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:12.487 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:12.486+0000 7ff3c4029780 -1 bdev(0x3b5ac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:12.487 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:12.486+0000 7ff3c4029780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:35:12.926 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:35:12.926 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:12.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:35:12.927 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:35:12.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:13.030 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:35:13.030 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:35:13.031 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:35:13.031 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:13.031 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:13.031 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:13.047 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:13.046+0000 7fc438a35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:13.048 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:13.048+0000 7fc438a35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:13.049 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:13.048+0000 7fc438a35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:13.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:35:13.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:13.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:35:13.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:13.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:13.132 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:13.132 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:13.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:13.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:13.133 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:13.231 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:13.230+0000 7fc438a35780 -1 Falling back to public interface 2026-02-13T19:35:13.233 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:13.666 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:13.665+0000 7fc438a35780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:14.234 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:14.235 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:14.235 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:14.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:14.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:14.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:14.333 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4248638911,v1:127.0.0.1:6801/4248638911] [v2:127.0.0.1:6802/4248638911,v1:127.0.0.1:6803/4248638911] exists,up f07a7981-35e9-41b0-bd30-2ae773b29f07 2026-02-13T19:35:14.334 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:14.334 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:14.335 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:14.335 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1456: test_get_osd_id_used_by_pgs: create_rbd_pool 2026-02-13T19:35:14.335 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:35:14.428 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:35:14.434 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:35:14.434 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:35:14.730 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:35:14.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:35:15.738 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:35:16.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1457: test_get_osd_id_used_by_pgs: wait_for_clean 2026-02-13T19:35:16.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:35:16.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:35:16.021 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:35:16.021 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:35:16.021 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:16.021 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:16.021 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:16.021 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:16.021 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:16.103 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:35:16.103 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:35:16.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:35:16.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:35:16.104 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:35:16.104 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:35:16.330 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:35:16.330 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:35:16.330 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:35:16.330 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:35:16.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:35:16.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:35:16.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:35:16.396 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:35:16.396 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:16.396 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:35:16.397 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:35:16.397 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:16.398 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:35:16.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:35:16.398 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:35:16.399 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:35:16.399 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:16.633 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:35:16.633 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:17.634 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:35:17.635 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:17.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:35:17.873 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:18.874 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:35:18.874 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:19.107 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:35:19.108 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:35:19.108 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:19.108 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:19.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:35:19.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:35:19.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:35:19.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:35:19.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:35:19.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:35:19.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:35:19.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:35:19.657 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:35:19.657 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:35:19.657 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:19.657 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:19.980 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:35:19.981 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:35:19.981 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:35:19.982 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: get_osd_id_used_by_pgs 2026-02-13T19:35:19.982 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: uniq 2026-02-13T19:35:19.982 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: ceph --format json pg dump pgs 2026-02-13T19:35:19.982 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: jq '.pg_stats | .[] | .up[], .acting[]' 2026-02-13T19:35:19.982 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: sort 2026-02-13T19:35:20.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: local osd_ids=0 2026-02-13T19:35:20.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1459: test_get_osd_id_used_by_pgs: test 0 = 0 2026-02-13T19:35:20.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1460: test_get_osd_id_used_by_pgs: teardown td/ceph-helpers 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:20.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:20.315 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:20.316 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:20.316 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:20.317 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:20.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:20.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:20.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:20.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:20.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:20.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:20.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:20.321 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:20.322 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:20.322 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:20.326 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:20.326 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.326 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.326 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:20.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:20.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:20.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:20.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osds td/ceph-helpers 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: test_get_osds: local dir=td/ceph-helpers 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1024: test_get_osds: setup td/ceph-helpers 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:20.329 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:20.330 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:20.330 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:20.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:20.331 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:20.332 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:20.332 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:20.332 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:20.333 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:20.333 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:20.333 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:20.333 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:20.334 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:20.335 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:20.335 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:20.336 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:20.336 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.336 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.336 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:20.337 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:20.337 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:20.337 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:35:20.338 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:35:20.338 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.338 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.338 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:35:20.339 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:35:20.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:35:20.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:35:20.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:35:20.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: test_get_osds: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-13T19:35:20.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:35:20.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:35:20.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:35:20.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:35:20.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:35:20.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-13T19:35:20.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:35:20.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:20.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:20.359 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:20.359 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.359 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.359 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:20.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-13T19:35:20.383 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:35:20.384 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:35:20.384 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:20.385 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:20.385 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:35:20.385 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:35:20.385 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:20.385 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:20.385 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:20.386 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:20.386 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.386 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.386 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:20.386 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:20.386 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:35:20.429 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:35:20.429 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:20.430 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:20.430 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:35:20.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:35:20.431 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:20.431 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:20.431 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:20.431 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:20.432 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.432 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.432 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:20.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:20.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:35:20.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1026: test_get_osds: run_mgr td/ceph-helpers x 2026-02-13T19:35:20.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:35:20.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:35:20.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:35:20.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:35:20.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:35:20.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:35:20.580 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:35:20.580 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:20.580 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:20.580 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:20.580 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.580 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.580 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:20.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:20.581 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:20.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: test_get_osds: run_osd td/ceph-helpers 0 2026-02-13T19:35:20.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:20.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:20.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:35:20.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:20.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:35:20.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:20.600 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:20.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:20.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:35:20.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:35:20.602 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:20.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fc0b1b34-d85d-4ebf-97eb-fe68f57c328a 2026-02-13T19:35:20.602 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fc0b1b34-d85d-4ebf-97eb-fe68f57c328a' 2026-02-13T19:35:20.603 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 fc0b1b34-d85d-4ebf-97eb-fe68f57c328a 2026-02-13T19:35:20.603 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:20.611 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD4fI9pA0RaJBAAWv4bDuE/hrmEMiKSr3QN/w== 2026-02-13T19:35:20.611 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD4fI9pA0RaJBAAWv4bDuE/hrmEMiKSr3QN/w=="}' 2026-02-13T19:35:20.611 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fc0b1b34-d85d-4ebf-97eb-fe68f57c328a -i td/ceph-helpers/0/new.json 2026-02-13T19:35:20.715 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:20.722 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:35:20.723 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD4fI9pA0RaJBAAWv4bDuE/hrmEMiKSr3QN/w== --osd-uuid fc0b1b34-d85d-4ebf-97eb-fe68f57c328a 2026-02-13T19:35:20.739 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:20.738+0000 7f035cc30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:20.740 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:20.739+0000 7f035cc30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:20.741 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:20.740+0000 7f035cc30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:20.742 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:20.740+0000 7f035cc30780 -1 bdev(0x34eac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:20.742 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:20.740+0000 7f035cc30780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:35:21.169 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:35:21.169 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:21.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:35:21.170 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:35:21.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:21.272 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:35:21.273 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:35:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:35:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:21.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:21.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:21.287 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:21.286+0000 7f8705427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:21.288 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:21.287+0000 7f8705427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:21.289 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:21.288+0000 7f8705427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:21.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:35:21.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:21.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:35:21.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:21.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:21.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:21.372 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:21.373 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:21.373 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:21.373 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:21.470 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:21.504 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:21.503+0000 7f8705427780 -1 Falling back to public interface 2026-02-13T19:35:21.791 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:21.789+0000 7f8705427780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:22.471 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:22.471 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:22.472 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:22.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:22.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:22.473 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:22.571 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3974150398,v1:127.0.0.1:6801/3974150398] [v2:127.0.0.1:6802/3974150398,v1:127.0.0.1:6803/3974150398] exists,up fc0b1b34-d85d-4ebf-97eb-fe68f57c328a 2026-02-13T19:35:22.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:22.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:22.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:22.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1028: test_get_osds: run_osd td/ceph-helpers 1 2026-02-13T19:35:22.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:22.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:22.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:22.576 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:22.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b2ebde67-7c29-4b95-a962-df6a6a59794e 2026-02-13T19:35:22.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b2ebde67-7c29-4b95-a962-df6a6a59794e' 2026-02-13T19:35:22.579 INFO:tasks.workunit.client.0.trial162.stdout:add osd1 b2ebde67-7c29-4b95-a962-df6a6a59794e 2026-02-13T19:35:22.579 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:22.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD6fI9plJ7VIhAACb16DlycvG+xuqOwzpsKkw== 2026-02-13T19:35:22.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD6fI9plJ7VIhAACb16DlycvG+xuqOwzpsKkw=="}' 2026-02-13T19:35:22.585 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b2ebde67-7c29-4b95-a962-df6a6a59794e -i td/ceph-helpers/1/new.json 2026-02-13T19:35:22.690 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:22.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-13T19:35:22.696 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD6fI9plJ7VIhAACb16DlycvG+xuqOwzpsKkw== --osd-uuid b2ebde67-7c29-4b95-a962-df6a6a59794e 2026-02-13T19:35:22.713 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:22.712+0000 7fba40c48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:22.715 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:22.714+0000 7fba40c48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:22.716 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:22.715+0000 7fba40c48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:22.716 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:22.715+0000 7fba40c48780 -1 bdev(0x351cc00 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:22.716 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:22.715+0000 7fba40c48780 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-13T19:35:23.038 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-13T19:35:23.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:23.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:35:23.039 INFO:tasks.workunit.client.0.trial162.stdout:adding osd1 key to auth repository 2026-02-13T19:35:23.040 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:23.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:35:23.364 INFO:tasks.workunit.client.0.trial162.stdout:start osd.1 2026-02-13T19:35:23.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:35:23.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:23.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:23.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:23.378 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:23.377+0000 7f49c9432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:23.379 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:23.378+0000 7f49c9432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:23.380 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:23.379+0000 7f49c9432780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:23.519 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:23.518+0000 7f49c9432780 -1 Falling back to public interface 2026-02-13T19:35:23.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:35:23.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:23.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:35:23.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:23.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:23.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:23.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:23.600 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:23.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:23.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:35:23.829 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:23.851 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:23.850+0000 7f49c9432780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:35:24.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:24.830 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:24.831 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:24.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:24.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:24.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:35:25.063 INFO:tasks.workunit.client.0.trial162.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/255933682,v1:127.0.0.1:6811/255933682] [v2:127.0.0.1:6812/255933682,v1:127.0.0.1:6813/255933682] exists,up b2ebde67-7c29-4b95-a962-df6a6a59794e 2026-02-13T19:35:25.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:25.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:25.064 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:25.064 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1029: test_get_osds: create_rbd_pool 2026-02-13T19:35:25.064 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:35:25.289 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:35:25.296 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:35:25.296 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:35:25.541 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:35:25.549 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:35:26.550 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:35:26.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1030: test_get_osds: wait_for_clean 2026-02-13T19:35:26.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:35:26.831 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:35:26.832 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:35:26.832 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:35:26.832 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:26.832 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:26.832 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:26.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:26.833 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:26.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:35:26.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:35:26.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:35:26.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:35:26.915 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:35:26.915 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:35:27.142 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:35:27.142 INFO:tasks.workunit.client.0.trial162.stderr:1' 2026-02-13T19:35:27.142 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:35:27.142 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:35:27.142 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:35:27.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:35:27.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:35:27.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:35:27.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:35:27.210 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:35:27.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-13T19:35:27.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-13T19:35:27.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483 1-42949672963' 2026-02-13T19:35:27.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:35:27.277 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:27.278 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:35:27.278 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:35:27.279 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:27.279 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:35:27.279 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:35:27.280 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:35:27.280 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:35:27.280 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:27.507 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:35:27.507 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:28.508 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:35:28.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:28.736 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836483 2026-02-13T19:35:28.736 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:35:28.737 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672963 2026-02-13T19:35:28.737 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:35:28.737 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:35:28.738 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-13T19:35:28.738 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:35:28.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672963 2026-02-13T19:35:28.739 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-13T19:35:28.739 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.1 seq 42949672963 2026-02-13T19:35:28.740 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:35:28.966 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672963 -lt 42949672963 2026-02-13T19:35:28.966 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:35:28.966 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:28.967 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:29.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:35:29.290 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:35:29.290 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:35:29.290 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:35:29.290 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:35:29.290 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:35:29.290 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:35:29.290 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:35:29.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:35:29.514 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:35:29.515 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:29.515 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:29.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:35:29.834 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:35:29.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:35:29.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: test_get_osds: create_rbd_pool 2026-02-13T19:35:29.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:35:30.112 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:35:30.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:35:30.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:35:30.402 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:35:30.411 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:35:31.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:35:31.694 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: get_osds rbd GROUP 2026-02-13T19:35:31.694 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=rbd 2026-02-13T19:35:31.694 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=GROUP 2026-02-13T19:35:31.694 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: grep --quiet '^[0-1] [0-1]$' 2026-02-13T19:35:31.694 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map rbd GROUP 2026-02-13T19:35:31.695 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:35:31.927 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:0' 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1033: test_get_osds: teardown td/ceph-helpers 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:31.928 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:32.034 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:32.035 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:32.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:32.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:32.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:32.037 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:32.038 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:32.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:32.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:32.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:32.039 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:32.040 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:32.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:32.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:32.046 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:32.046 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.047 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.047 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:32.048 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_pg td/ceph-helpers 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1091: test_get_pg: local dir=td/ceph-helpers 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: test_get_pg: setup td/ceph-helpers 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:32.049 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:32.050 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:32.050 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:32.050 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:32.050 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:32.050 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:32.050 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:32.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:32.051 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:32.052 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:32.052 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:32.052 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:32.053 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:32.053 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:32.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:32.054 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:32.054 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:32.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:32.055 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:32.056 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:32.056 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.057 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:32.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:32.057 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:32.058 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:35:32.059 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:35:32.059 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.059 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.059 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:35:32.060 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:35:32.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:35:32.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:35:32.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:35:32.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: test_get_pg: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:32.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:35:32.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:35:32.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:35:32.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:35:32.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:35:32.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:32.081 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:35:32.081 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:32.081 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:32.081 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:32.081 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.081 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.081 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:32.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:32.106 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:32.107 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:32.108 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:32.108 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.108 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.108 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:32.108 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:32.108 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:35:32.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:35:32.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:32.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:32.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:35:32.151 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:32.152 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:35:32.195 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1095: test_get_pg: run_mgr td/ceph-helpers x 2026-02-13T19:35:32.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:35:32.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:35:32.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:35:32.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:35:32.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:35:32.196 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:35:32.298 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:35:32.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:32.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:32.299 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:32.299 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.299 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.299 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:32.300 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:32.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: test_get_pg: run_osd td/ceph-helpers 0 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:32.317 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:32.318 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:32.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:35:32.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:35:32.320 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:32.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dbe9417e-daaf-414d-9977-145729758cab 2026-02-13T19:35:32.320 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dbe9417e-daaf-414d-9977-145729758cab' 2026-02-13T19:35:32.321 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 dbe9417e-daaf-414d-9977-145729758cab 2026-02-13T19:35:32.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:32.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAEfY9pbYCCExAAP7cpj+TtMVy0vQF82fytpg== 2026-02-13T19:35:32.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAEfY9pbYCCExAAP7cpj+TtMVy0vQF82fytpg=="}' 2026-02-13T19:35:32.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dbe9417e-daaf-414d-9977-145729758cab -i td/ceph-helpers/0/new.json 2026-02-13T19:35:32.431 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:32.439 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:35:32.440 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAEfY9pbYCCExAAP7cpj+TtMVy0vQF82fytpg== --osd-uuid dbe9417e-daaf-414d-9977-145729758cab 2026-02-13T19:35:32.455 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:32.454+0000 7f6392233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:32.457 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:32.456+0000 7f6392233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:32.458 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:32.457+0000 7f6392233780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:32.458 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:32.457+0000 7f6392233780 -1 bdev(0x34b6c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:32.459 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:32.457+0000 7f6392233780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:35:32.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:35:32.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:32.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:35:32.883 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:35:32.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:32.986 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:35:32.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:35:32.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:35:32.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:32.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:32.987 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:33.001 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:33.000+0000 7fdf97e5f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:33.002 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:33.002+0000 7fdf97e5f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:33.003 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:33.003+0000 7fdf97e5f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:33.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:35:33.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:33.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:35:33.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:33.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:33.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:33.088 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:33.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:33.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:33.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:33.185 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:33.228 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:33.227+0000 7fdf97e5f780 -1 Falling back to public interface 2026-02-13T19:35:33.547 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:33.546+0000 7fdf97e5f780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:34.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:34.186 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:34.187 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:34.187 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:34.187 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:34.187 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:34.288 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/437471946,v1:127.0.0.1:6801/437471946] [v2:127.0.0.1:6802/437471946,v1:127.0.0.1:6803/437471946] exists,up dbe9417e-daaf-414d-9977-145729758cab 2026-02-13T19:35:34.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:34.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:34.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:34.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1097: test_get_pg: create_rbd_pool 2026-02-13T19:35:34.289 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:35:34.384 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:35:34.390 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:35:34.390 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:35:34.623 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:35:34.631 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:35:35.633 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:35:35.912 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1098: test_get_pg: wait_for_clean 2026-02-13T19:35:35.913 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:35:35.913 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:35:35.913 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:35:35.913 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:35:35.913 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:35.913 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:35.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:35.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:35.914 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:35.996 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:35:35.996 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:35:35.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:35:35.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:35:35.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:35:35.997 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:35:36.231 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:35:36.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:35:36.232 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:35:36.232 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:35:36.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:35:36.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:35:36.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:35:36.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:35:36.300 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:36.300 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:35:36.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:35:36.301 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:36.302 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:35:36.303 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:35:36.303 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:35:36.303 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:35:36.303 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:36.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-13T19:35:36.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:37.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:35:37.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:37.769 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836483 2026-02-13T19:35:37.770 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:38.770 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:35:38.771 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:38.999 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:35:39.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:35:39.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:39.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:39.322 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:35:39.322 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:35:39.323 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:35:39.323 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:35:39.323 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:35:39.323 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:35:39.323 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:35:39.323 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:35:39.548 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:35:39.548 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:35:39.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:39.549 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:39.864 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: get_pg rbd GROUP 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=GROUP 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: grep --quiet '^[0-9]\.[0-9a-f][0-9a-f]*$' 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd GROUP 2026-02-13T19:35:39.865 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-13T19:35:40.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1100: test_get_pg: teardown td/ceph-helpers 2026-02-13T19:35:40.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:40.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:40.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:40.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:40.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:40.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:40.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:40.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:40.205 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:40.205 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:40.206 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:40.206 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:40.208 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:40.208 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:40.208 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:40.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:40.209 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:40.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:40.210 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:40.210 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:40.211 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:40.211 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:40.215 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:40.215 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.215 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.215 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:40.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:40.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:40.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:40.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_primary td/ceph-helpers 2026-02-13T19:35:40.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: test_get_primary: local dir=td/ceph-helpers 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: test_get_primary: setup td/ceph-helpers 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:40.218 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:40.219 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:40.219 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:40.219 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:40.220 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:40.220 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:40.220 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:40.221 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:40.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:40.221 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:40.222 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:40.222 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:40.223 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:40.224 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:40.224 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:40.225 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:40.225 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.225 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.226 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:40.226 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:40.226 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:40.227 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:35:40.227 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:35:40.227 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.227 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.228 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:35:40.228 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:35:40.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: test_get_primary: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:35:40.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:40.247 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:35:40.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:40.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:40.248 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:40.248 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.248 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.248 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:40.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:40.273 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:40.275 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:40.276 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.276 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.276 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:40.276 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:40.276 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:35:40.318 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.319 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.320 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:40.320 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:40.320 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:35:40.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: test_get_primary: local osd=0 2026-02-13T19:35:40.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: test_get_primary: run_mgr td/ceph-helpers x 2026-02-13T19:35:40.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:35:40.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:35:40.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:35:40.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:35:40.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:35:40.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:35:40.460 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:35:40.460 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:40.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:40.461 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:40.461 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.461 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:40.462 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:40.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:40.480 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1201: test_get_primary: run_osd td/ceph-helpers 0 2026-02-13T19:35:40.480 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:40.480 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:40.480 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:40.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:40.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:35:40.483 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:40.484 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=65f92fa9-7ccd-4df2-b37c-20b129953540 2026-02-13T19:35:40.484 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 65f92fa9-7ccd-4df2-b37c-20b129953540' 2026-02-13T19:35:40.484 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 65f92fa9-7ccd-4df2-b37c-20b129953540 2026-02-13T19:35:40.484 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:40.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAMfY9phT4vHRAAB/ZOz+LDWhaSPythy1OM7Q== 2026-02-13T19:35:40.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAMfY9phT4vHRAAB/ZOz+LDWhaSPythy1OM7Q=="}' 2026-02-13T19:35:40.490 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 65f92fa9-7ccd-4df2-b37c-20b129953540 -i td/ceph-helpers/0/new.json 2026-02-13T19:35:40.592 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:40.598 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:35:40.599 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMfY9phT4vHRAAB/ZOz+LDWhaSPythy1OM7Q== --osd-uuid 65f92fa9-7ccd-4df2-b37c-20b129953540 2026-02-13T19:35:40.615 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:40.613+0000 7f5884c31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:40.616 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:40.615+0000 7f5884c31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:40.617 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:40.616+0000 7f5884c31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:40.618 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:40.616+0000 7f5884c31780 -1 bdev(0x3dd0c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:40.618 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:40.616+0000 7f5884c31780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:35:41.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:35:41.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:41.020 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:35:41.021 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:35:41.021 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:41.126 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:35:41.127 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:35:41.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:35:41.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:41.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:41.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:41.142 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:41.141+0000 7fa036a52780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:41.143 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:41.142+0000 7fa036a52780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:41.144 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:41.143+0000 7fa036a52780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:41.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:35:41.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:41.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:35:41.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:41.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:41.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:41.229 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:41.230 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:41.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:41.230 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:41.328 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:41.332 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:41.331+0000 7fa036a52780 -1 Falling back to public interface 2026-02-13T19:35:41.695 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:41.694+0000 7fa036a52780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:42.330 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:42.330 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:42.330 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:42.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:42.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:42.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:42.429 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1622532302,v1:127.0.0.1:6801/1622532302] [v2:127.0.0.1:6802/1622532302,v1:127.0.0.1:6803/1622532302] exists,up 65f92fa9-7ccd-4df2-b37c-20b129953540 2026-02-13T19:35:42.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:42.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:42.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:42.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: test_get_primary: create_rbd_pool 2026-02-13T19:35:42.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:35:42.526 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:35:42.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:35:42.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:35:42.714 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:35:42.720 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:35:43.721 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:35:43.999 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: test_get_primary: wait_for_clean 2026-02-13T19:35:43.999 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:35:43.999 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:35:43.999 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:35:44.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:35:44.000 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:44.000 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:44.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:44.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:44.000 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:44.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:35:44.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:35:44.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:35:44.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:35:44.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:35:44.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:35:44.318 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:35:44.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:35:44.319 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:35:44.319 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:35:44.386 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:35:44.387 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:35:44.387 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:35:44.387 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:35:44.387 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:44.387 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:35:44.388 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:35:44.388 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:44.388 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:35:44.389 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:35:44.389 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:35:44.389 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:35:44.389 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:44.619 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:35:44.619 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:35:44.620 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:44.620 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:44.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:35:44.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:35:44.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:35:44.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:35:44.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:35:44.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:35:44.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:35:44.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:35:45.173 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:35:45.174 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:35:45.174 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:45.174 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: get_primary rbd GROUP 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-13T19:35:45.498 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T19:35:45.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: test 0 = 0 2026-02-13T19:35:45.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1205: test_get_primary: teardown td/ceph-helpers 2026-02-13T19:35:45.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:45.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:45.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:45.730 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:45.730 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:45.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:45.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:45.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:45.835 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:45.836 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:45.837 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:45.837 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:45.839 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:45.839 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:45.840 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:45.840 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:45.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:45.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:45.841 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:45.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:45.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:45.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:45.845 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:45.845 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:45.845 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:45.845 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:45.846 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:45.846 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:45.846 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:45.846 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_timeout_delays td/ceph-helpers 2026-02-13T19:35:45.847 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: get_timeout_delays 1 2026-02-13T19:35:45.847 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.847 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.847 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.847 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.847 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:45.857 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: test '1 ' = '1 ' 2026-02-13T19:35:45.859 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: get_timeout_delays 5 2026-02-13T19:35:45.859 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.859 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.859 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.860 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.860 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:45.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: test '1 2 2 ' = '1 2 2 ' 2026-02-13T19:35:45.877 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: get_timeout_delays 6 2026-02-13T19:35:45.877 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.877 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.877 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.877 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.877 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:45.896 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: test '1 2 3 ' = '1 2 3 ' 2026-02-13T19:35:45.896 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: get_timeout_delays 7 2026-02-13T19:35:45.897 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.897 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.897 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.897 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.897 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:45.918 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: test '1 2 4 ' = '1 2 4 ' 2026-02-13T19:35:45.919 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: get_timeout_delays 8 2026-02-13T19:35:45.919 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.920 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.920 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.920 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.920 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:45.943 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: test '1 2 4 1 ' = '1 2 4 1 ' 2026-02-13T19:35:45.944 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: get_timeout_delays 1 .1 2026-02-13T19:35:45.944 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.944 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.945 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:45.968 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: test '0.1 0.2 0.4 0.3 ' = '0.1 0.2 0.4 0.3 ' 2026-02-13T19:35:45.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: get_timeout_delays 1.5 .1 2026-02-13T19:35:45.969 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.969 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:45.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 ' = '0.1 0.2 0.4 0.8 ' 2026-02-13T19:35:45.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: get_timeout_delays 5 .1 2026-02-13T19:35:45.998 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:45.999 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:45.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:45.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:45.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:46.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 1.9 ' = '0.1 0.2 0.4 0.8 1.6 1.9 ' 2026-02-13T19:35:46.036 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: get_timeout_delays 6 .1 2026-02-13T19:35:46.037 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:46.037 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:46.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:46.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:46.037 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:46.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 2.9 ' = '0.1 0.2 0.4 0.8 1.6 2.9 ' 2026-02-13T19:35:46.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: get_timeout_delays 6.3 .1 2026-02-13T19:35:46.074 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:46.074 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:46.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:46.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:46.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:46.110 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 ' = '0.1 0.2 0.4 0.8 1.6 3.2 ' 2026-02-13T19:35:46.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: get_timeout_delays 20 .1 2026-02-13T19:35:46.111 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:46.111 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:46.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:46.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:46.111 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:46.160 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' 2026-02-13T19:35:46.161 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: get_timeout_delays 300 .1 0 2026-02-13T19:35:46.162 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:46.162 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:46.162 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:46.162 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:46.162 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:46.236 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' 2026-02-13T19:35:46.236 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: get_timeout_delays 300 .1 10 2026-02-13T19:35:46.237 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:46.237 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:46.237 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:46.237 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:46.238 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_is_clean td/ceph-helpers 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1552: test_is_clean: local dir=td/ceph-helpers 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1554: test_is_clean: setup td/ceph-helpers 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:46.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:46.464 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:46.464 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:46.465 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:46.465 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:46.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:46.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:46.466 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:46.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:46.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:46.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:46.468 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:46.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:46.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:46.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:46.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:46.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:46.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:46.471 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:46.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:46.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:46.472 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:35:46.473 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:35:46.473 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:46.473 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:46.473 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:35:46.474 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:35:46.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:35:46.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:35:46.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:35:46.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1555: test_is_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:46.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:35:46.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:35:46.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:35:46.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:35:46.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:35:46.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:46.494 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:35:46.494 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:46.494 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:46.494 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:46.494 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:46.494 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:46.494 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:46.495 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:46.518 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:46.521 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:46.522 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:46.522 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:46.522 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:46.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:46.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:35:46.563 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:35:46.563 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:46.563 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:46.563 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:35:46.563 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:35:46.563 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:46.563 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:46.564 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:46.564 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:46.564 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:46.564 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:46.564 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:46.564 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:46.564 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:35:46.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1556: test_is_clean: run_mgr td/ceph-helpers x 2026-02-13T19:35:46.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:35:46.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:35:46.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:35:46.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:35:46.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:35:46.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:35:46.709 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:35:46.709 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:46.709 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:46.709 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:46.709 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:46.710 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:46.710 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:46.710 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:46.710 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:46.729 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1557: test_is_clean: run_osd td/ceph-helpers 0 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:46.730 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:46.731 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:46.732 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9325cf9f-7f4d-4b1d-b9cb-b83d8fb7a9d9 2026-02-13T19:35:46.733 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9325cf9f-7f4d-4b1d-b9cb-b83d8fb7a9d9' 2026-02-13T19:35:46.733 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 9325cf9f-7f4d-4b1d-b9cb-b83d8fb7a9d9 2026-02-13T19:35:46.733 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:46.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQASfY9pMmsTLBAAt55ue8Rp3CeHG9Go85bErA== 2026-02-13T19:35:46.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQASfY9pMmsTLBAAt55ue8Rp3CeHG9Go85bErA=="}' 2026-02-13T19:35:46.740 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9325cf9f-7f4d-4b1d-b9cb-b83d8fb7a9d9 -i td/ceph-helpers/0/new.json 2026-02-13T19:35:46.848 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:46.855 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:35:46.856 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQASfY9pMmsTLBAAt55ue8Rp3CeHG9Go85bErA== --osd-uuid 9325cf9f-7f4d-4b1d-b9cb-b83d8fb7a9d9 2026-02-13T19:35:46.872 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:46.871+0000 7f2eee243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:46.874 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:46.873+0000 7f2eee243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:46.875 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:46.873+0000 7f2eee243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:46.875 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:46.874+0000 7f2eee243780 -1 bdev(0x2ba2c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:46.875 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:46.874+0000 7f2eee243780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:35:47.361 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:35:47.361 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:47.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:35:47.362 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:35:47.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:47.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:35:47.468 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:35:47.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:35:47.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:47.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:47.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:47.483 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:47.482+0000 7ff6e1e27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:47.484 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:47.483+0000 7ff6e1e27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:47.485 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:47.484+0000 7ff6e1e27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:47.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:35:47.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:47.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:35:47.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:47.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:47.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:47.567 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:47.567 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:47.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:47.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:47.665 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:47.715 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:47.713+0000 7ff6e1e27780 -1 Falling back to public interface 2026-02-13T19:35:48.032 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:48.031+0000 7ff6e1e27780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:48.666 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:48.666 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:48.667 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:48.667 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:48.667 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:48.667 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:48.768 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1657114011,v1:127.0.0.1:6801/1657114011] [v2:127.0.0.1:6802/1657114011,v1:127.0.0.1:6803/1657114011] exists,up 9325cf9f-7f4d-4b1d-b9cb-b83d8fb7a9d9 2026-02-13T19:35:48.769 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:48.769 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:48.769 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:48.769 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1558: test_is_clean: create_rbd_pool 2026-02-13T19:35:48.769 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:35:48.865 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:35:48.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:35:48.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:35:49.192 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:35:49.199 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:35:50.201 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:35:50.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1559: test_is_clean: wait_for_clean 2026-02-13T19:35:50.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:35:50.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:35:50.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:35:50.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:35:50.480 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:35:50.480 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:35:50.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:35:50.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:35:50.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:35:50.561 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:35:50.562 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:35:50.562 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:35:50.562 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:35:50.562 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:35:50.562 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:35:50.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:35:50.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:35:50.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:35:50.794 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:35:50.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:35:50.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:35:50.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:35:50.859 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:35:50.859 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:50.859 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:35:50.860 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:35:50.860 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:35:50.861 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:35:50.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:35:50.862 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:35:50.862 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:35:50.862 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:51.097 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:35:51.097 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:52.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:35:52.098 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:52.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:35:52.331 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:35:53.332 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:35:53.333 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:35:53.562 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:35:53.562 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:35:53.563 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:53.563 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:53.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:35:53.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:35:53.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:35:53.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:35:53.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:35:53.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:35:53.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:35:53.885 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:35:54.108 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:35:54.108 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:35:54.108 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:54.109 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:54.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:35:54.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:35:54.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:35:54.425 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1560: test_is_clean: is_clean 2026-02-13T19:35:54.425 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: get_num_pgs 2026-02-13T19:35:54.425 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:35:54.425 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: num_pgs=4 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1547: is_clean: test 4 '!=' 0 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: get_num_active_clean 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:35:54.743 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:35:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: test 4 = 4 2026-02-13T19:35:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1561: test_is_clean: teardown td/ceph-helpers 2026-02-13T19:35:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:54.969 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:54.969 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:54.970 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:54.970 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:54.970 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:54.970 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:55.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:55.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:55.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:55.078 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:55.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:55.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:55.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:55.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:55.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:55.081 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:55.081 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:55.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:55.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:55.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:55.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:55.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemon td/ceph-helpers 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: test_kill_daemon: local dir=td/ceph-helpers 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:267: test_kill_daemon: setup td/ceph-helpers 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:35:55.088 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:35:55.089 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:35:55.089 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:35:55.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:35:55.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:35:55.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:35:55.090 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:35:55.091 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:35:55.091 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:35:55.092 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:35:55.092 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:35:55.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:35:55.093 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:35:55.093 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:55.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:35:55.094 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:35:55.094 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:35:55.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:35:55.095 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:35:55.096 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:35:55.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:35:55.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.097 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.097 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:35:55.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:35:55.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:35:55.098 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:35:55.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:35:55.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.099 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.099 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:35:55.100 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:35:55.101 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: test_kill_daemon: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:35:55.102 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:55.120 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:35:55.120 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:55.120 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:55.121 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:55.121 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.121 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.121 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:55.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:35:55.146 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:35:55.148 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.149 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:55.150 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:55.150 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:35:55.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:35:55.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:35:55.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:35:55.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:35:55.193 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:35:55.193 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:35:55.193 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:35:55.194 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:35:55.194 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:35:55.195 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.195 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.195 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:35:55.195 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:35:55.195 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:35:55.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: test_kill_daemon: run_mgr td/ceph-helpers x 2026-02-13T19:35:55.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:35:55.237 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:35:55.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:35:55.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:35:55.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:35:55.238 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:35:55.344 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:35:55.344 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:55.344 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:55.344 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:55.344 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.344 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.344 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:55.345 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:55.345 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:35:55.363 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:270: test_kill_daemon: run_osd td/ceph-helpers 0 2026-02-13T19:35:55.363 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:35:55.363 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:35:55.363 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:35:55.363 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:35:55.364 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:35:55.365 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:35:55.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:35:55.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:35:55.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:35:55.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:35:55.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:35:55.366 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:35:55.366 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:35:55.367 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0f060a3d-a80d-4b03-831f-eb7995090440 2026-02-13T19:35:55.367 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0f060a3d-a80d-4b03-831f-eb7995090440' 2026-02-13T19:35:55.367 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 0f060a3d-a80d-4b03-831f-eb7995090440 2026-02-13T19:35:55.367 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:35:55.373 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAbfY9p19A6FhAAgjmSUG0dpSN5dMNMzSxfSw== 2026-02-13T19:35:55.373 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAbfY9p19A6FhAAgjmSUG0dpSN5dMNMzSxfSw=="}' 2026-02-13T19:35:55.374 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0f060a3d-a80d-4b03-831f-eb7995090440 -i td/ceph-helpers/0/new.json 2026-02-13T19:35:55.479 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:55.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:35:55.486 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAbfY9p19A6FhAAgjmSUG0dpSN5dMNMzSxfSw== --osd-uuid 0f060a3d-a80d-4b03-831f-eb7995090440 2026-02-13T19:35:55.502 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:55.502+0000 7fcc2c015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:55.504 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:55.504+0000 7fcc2c015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:55.504 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:55.504+0000 7fcc2c015780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:55.505 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:55.505+0000 7fcc2c015780 -1 bdev(0x2a3ac00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:35:55.505 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:55.505+0000 7fcc2c015780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:35:55.939 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:35:55.939 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:35:55.940 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:35:55.940 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:35:55.940 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:35:56.044 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:35:56.044 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:35:56.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:35:56.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:35:56.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:35:56.045 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:35:56.060 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:56.060+0000 7f22f83e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:56.061 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:56.061+0000 7f22f83e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:56.062 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:56.062+0000 7f22f83e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:35:56.146 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:35:56.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:35:56.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:35:56.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:35:56.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:35:56.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:56.147 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:35:56.147 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:35:56.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:56.148 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:56.244 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:35:56.255 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:56.255+0000 7f22f83e3780 -1 Falling back to public interface 2026-02-13T19:35:56.577 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:35:56.577+0000 7f22f83e3780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:35:57.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:35:57.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:35:57.246 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:35:57.246 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:35:57.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:35:57.247 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:35:57.350 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/326230942,v1:127.0.0.1:6801/326230942] [v2:127.0.0.1:6802/326230942,v1:127.0.0.1:6803/326230942] exists,up 0f060a3d-a80d-4b03-831f-eb7995090440 2026-02-13T19:35:57.351 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:35:57.352 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:35:57.352 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:35:57.352 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:272: test_kill_daemon: name_prefix=osd 2026-02-13T19:35:57.352 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: find td/ceph-helpers 2026-02-13T19:35:57.352 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: grep 'osd[^/]*\.pid' 2026-02-13T19:35:57.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-13T19:35:57.353 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:280: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid 0 1 2026-02-13T19:35:57.353 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=116402 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=0 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local delays=1 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 116402 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T19:35:57.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 1 2026-02-13T19:35:58.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 116402 2026-02-13T19:35:58.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T19:35:58.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T19:35:58.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0 2026-02-13T19:35:58.357 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 1 2026-02-13T19:35:58.357 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:284: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid TERM 2026-02-13T19:35:58.357 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=116402 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 116402 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T19:35:58.359 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-13T19:35:58.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 116402 2026-02-13T19:35:58.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-13T19:35:58.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-13T19:35:58.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-13T19:35:58.461 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:287: test_kill_daemon: name_prefix=mgr 2026-02-13T19:35:58.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: find td/ceph-helpers 2026-02-13T19:35:58.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: grep 'mgr[^/]*\.pid' 2026-02-13T19:35:58.461 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-13T19:35:58.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:292: test_kill_daemon: kill_daemon td/ceph-helpers/mgr.x.pid TERM 2026-02-13T19:35:58.462 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mgr.x.pid 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=115547 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 115547 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T19:35:58.463 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T19:35:58.464 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-13T19:35:58.564 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.564 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 115547 2026-02-13T19:35:58.564 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-13T19:35:58.564 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-13T19:35:58.564 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-13T19:35:58.565 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:295: test_kill_daemon: name_prefix=mon 2026-02-13T19:35:58.565 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: find td/ceph-helpers 2026-02-13T19:35:58.565 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: grep 'mon[^/]*\.pid' 2026-02-13T19:35:58.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-13T19:35:58.566 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:300: test_kill_daemon: kill_daemon td/ceph-helpers/mon.a.pid TERM 2026-02-13T19:35:58.566 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mon.a.pid 2026-02-13T19:35:58.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=115462 2026-02-13T19:35:58.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-13T19:35:58.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-13T19:35:58.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-13T19:35:58.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 115462 2026-02-13T19:35:58.568 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T19:35:58.569 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T19:35:58.569 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-13T19:35:58.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 115462 2026-02-13T19:35:58.669 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-13T19:35:58.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-13T19:35:58.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.2 2026-02-13T19:35:58.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-13T19:35:58.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 115462 2026-02-13T19:35:58.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-13T19:35:58.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-13T19:35:58.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-13T19:35:58.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:301: test_kill_daemon: timeout 5 ceph status 2026-02-13T19:36:03.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:304: test_kill_daemon: teardown td/ceph-helpers 2026-02-13T19:36:03.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:03.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:03.875 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:03.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:03.876 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:03.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:03.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:03.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:03.880 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:03.881 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:03.881 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:03.882 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:03.882 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:03.883 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:03.883 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:03.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:03.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:03.884 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:03.884 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:03.885 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:03.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:03.886 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:03.889 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:03.890 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:03.890 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:03.890 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemons td/ceph-helpers 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:357: test_kill_daemons: local dir=td/ceph-helpers 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:358: test_kill_daemons: setup td/ceph-helpers 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:03.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:03.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:03.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:03.893 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:03.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:03.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:03.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:03.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:03.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:03.894 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:03.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:03.895 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:03.895 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:03.895 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:03.896 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:03.896 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:03.896 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:03.896 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:03.897 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:03.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:03.898 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:03.899 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:03.899 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:03.900 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:03.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:03.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:03.900 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:03.901 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:36:03.902 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:36:03.902 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:03.902 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:03.902 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:36:03.903 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:359: test_kill_daemons: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:36:03.904 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:36:03.905 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:03.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:36:03.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:03.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:03.922 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:03.922 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:03.922 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:03.922 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:03.923 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:03.946 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:03.948 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:03.949 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:03.949 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:03.949 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:03.949 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:03.949 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:03.949 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:36:03.991 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:36:03.991 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:03.991 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:03.991 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:36:03.991 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:36:03.992 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:03.992 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:03.992 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:03.993 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:03.993 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:03.993 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:03.993 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:03.993 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:03.993 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:36:04.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:360: test_kill_daemons: run_mgr td/ceph-helpers x 2026-02-13T19:36:04.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:36:04.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:36:04.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:36:04.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:36:04.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:36:04.035 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:36:04.133 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:36:04.133 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:04.133 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:04.134 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:04.134 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:04.134 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:04.134 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:04.134 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:36:04.135 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:361: test_kill_daemons: run_osd td/ceph-helpers 0 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:36:04.153 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:04.154 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:36:04.155 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:36:04.156 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:36:04.156 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:36:04.156 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0132f3fd-5ed8-4864-892f-69d9ca6b8b3c 2026-02-13T19:36:04.156 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0132f3fd-5ed8-4864-892f-69d9ca6b8b3c' 2026-02-13T19:36:04.156 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 0132f3fd-5ed8-4864-892f-69d9ca6b8b3c 2026-02-13T19:36:04.156 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:36:04.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAkfY9p2py5CRAAx9TLX5mm+qhVTw/q2kUoRA== 2026-02-13T19:36:04.163 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAkfY9p2py5CRAAx9TLX5mm+qhVTw/q2kUoRA=="}' 2026-02-13T19:36:04.164 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0132f3fd-5ed8-4864-892f-69d9ca6b8b3c -i td/ceph-helpers/0/new.json 2026-02-13T19:36:04.268 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:04.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:36:04.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAkfY9p2py5CRAAx9TLX5mm+qhVTw/q2kUoRA== --osd-uuid 0132f3fd-5ed8-4864-892f-69d9ca6b8b3c 2026-02-13T19:36:04.291 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.291+0000 7f58a89c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:04.293 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.293+0000 7f58a89c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:04.294 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.294+0000 7f58a89c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:04.294 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.294+0000 7f58a89c6780 -1 bdev(0x27b4c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:36:04.294 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.294+0000 7f58a89c6780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:36:04.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:36:04.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:36:04.717 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:36:04.718 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:36:04.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:36:04.822 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:36:04.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:36:04.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:36:04.823 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:36:04.824 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:36:04.824 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:36:04.838 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.838+0000 7fccbcc48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:04.840 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.840+0000 7fccbcc48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:04.841 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:04.841+0000 7fccbcc48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:04.924 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:05.025 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:36:05.044 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:05.045+0000 7fccbcc48780 -1 Falling back to public interface 2026-02-13T19:36:05.410 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:05.409+0000 7fccbcc48780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:36:06.026 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:36:06.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:06.027 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:36:06.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:36:06.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:06.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/662283427,v1:127.0.0.1:6801/662283427] [v2:127.0.0.1:6802/662283427,v1:127.0.0.1:6803/662283427] exists,up 0132f3fd-5ed8-4864-892f-69d9ca6b8b3c 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:368: test_kill_daemons: kill_daemons td/ceph-helpers 0 osd 1 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:06.128 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:07.134 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 1 2026-02-13T19:36:07.135 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:372: test_kill_daemons: kill_daemons td/ceph-helpers TERM osd 2026-02-13T19:36:07.135 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:07.136 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:07.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:07.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:07.136 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:07.241 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:07.241 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:376: test_kill_daemons: kill_daemons td/ceph-helpers TERM mgr 2026-02-13T19:36:07.242 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:07.242 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:07.242 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:07.243 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:07.243 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:07.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:07.348 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:380: test_kill_daemons: kill_daemons td/ceph-helpers TERM 2026-02-13T19:36:07.349 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:07.349 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:07.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:07.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:07.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:07.456 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:07.457 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:381: test_kill_daemons: timeout 5 ceph status 2026-02-13T19:36:12.460 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:382: test_kill_daemons: teardown td/ceph-helpers 2026-02-13T19:36:12.461 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:12.461 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:12.461 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:12.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:12.461 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:12.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:12.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:12.462 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:12.466 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:12.466 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:12.467 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:12.467 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:12.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:12.468 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:12.468 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:12.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:12.469 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:12.470 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:12.470 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:12.470 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:12.471 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:12.471 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:12.474 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:12.474 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.474 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.474 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:12.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:12.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:12.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_objectstore_tool td/ceph-helpers 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: test_objectstore_tool: local dir=td/ceph-helpers 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: test_objectstore_tool: setup td/ceph-helpers 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:12.477 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:12.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:12.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:12.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:12.478 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:12.478 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:12.479 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:12.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:12.479 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:12.480 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:12.480 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:12.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:12.481 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:12.481 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:12.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:12.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:12.483 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:12.484 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:12.484 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.484 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.484 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:12.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:12.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:12.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:36:12.486 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:36:12.486 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.486 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:36:12.487 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:36:12.488 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1301: test_objectstore_tool: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:36:12.489 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:12.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:36:12.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:12.508 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:12.509 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:12.509 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.509 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.509 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:12.509 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:12.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:12.535 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:12.536 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:12.536 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.537 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.537 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:12.537 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:12.537 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:36:12.580 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:36:12.580 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:12.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:12.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:36:12.581 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:36:12.582 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:12.582 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:12.582 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:12.582 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:12.582 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.582 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.583 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:12.583 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:12.583 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:36:12.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: test_objectstore_tool: local osd=0 2026-02-13T19:36:12.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: test_objectstore_tool: run_mgr td/ceph-helpers x 2026-02-13T19:36:12.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:36:12.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:36:12.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:36:12.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:36:12.625 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:36:12.626 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:36:12.729 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:36:12.729 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:12.729 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:12.729 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:12.729 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.729 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:12.757 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:36:12.757 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:36:12.757 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: test_objectstore_tool: run_osd td/ceph-helpers 0 2026-02-13T19:36:12.757 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:36:12.757 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:36:12.758 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:36:12.759 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=853e72cf-5304-4725-93d8-03c98765817b 2026-02-13T19:36:12.760 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 853e72cf-5304-4725-93d8-03c98765817b' 2026-02-13T19:36:12.761 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 853e72cf-5304-4725-93d8-03c98765817b 2026-02-13T19:36:12.761 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:36:12.761 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAsfY9pjEktLRAAXUAZ+469CieNqRNVe4Q0Lg== 2026-02-13T19:36:12.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAsfY9pjEktLRAAXUAZ+469CieNqRNVe4Q0Lg=="}' 2026-02-13T19:36:12.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 853e72cf-5304-4725-93d8-03c98765817b -i td/ceph-helpers/0/new.json 2026-02-13T19:36:12.864 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:12.871 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:36:12.872 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAsfY9pjEktLRAAXUAZ+469CieNqRNVe4Q0Lg== --osd-uuid 853e72cf-5304-4725-93d8-03c98765817b 2026-02-13T19:36:12.888 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:12.887+0000 7f9d12813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:12.890 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:12.889+0000 7f9d12813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:12.890 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:12.889+0000 7f9d12813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:12.891 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:12.890+0000 7f9d12813780 -1 bdev(0x4104c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:36:12.891 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:12.890+0000 7f9d12813780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:36:13.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:36:13.300 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:36:13.301 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:36:13.301 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:36:13.302 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:36:13.405 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:36:13.405 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:36:13.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:36:13.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:36:13.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:36:13.406 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:36:13.420 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:13.419+0000 7fc8b3dfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:13.421 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:13.420+0000 7fc8b3dfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:13.422 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:13.421+0000 7fc8b3dfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:13.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:36:13.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:36:13.506 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:36:13.507 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:36:13.507 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:36:13.507 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:13.507 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:13.508 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:36:13.508 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:13.508 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:13.605 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:36:13.624 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:13.622+0000 7fc8b3dfd780 -1 Falling back to public interface 2026-02-13T19:36:13.943 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:13.941+0000 7fc8b3dfd780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:36:14.606 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:36:14.606 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:14.607 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:36:14.607 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:36:14.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:14.608 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:14.707 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2875411580,v1:127.0.0.1:6801/2875411580] [v2:127.0.0.1:6802/2875411580,v1:127.0.0.1:6803/2875411580] exists,up 853e72cf-5304-4725-93d8-03c98765817b 2026-02-13T19:36:14.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:36:14.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:36:14.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:36:14.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: test_objectstore_tool: create_rbd_pool 2026-02-13T19:36:14.707 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:36:14.804 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:36:14.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:36:14.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:36:15.112 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:36:15.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:36:16.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:36:16.401 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: test_objectstore_tool: wait_for_clean 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:36:16.402 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:36:16.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:36:16.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:36:16.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:36:16.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:36:16.485 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:36:16.485 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:36:16.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:36:16.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:36:16.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:36:16.717 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:36:16.779 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:36:16.779 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:36:16.779 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:36:16.779 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:36:16.780 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:36:16.780 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:36:16.781 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:36:16.781 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:36:16.781 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:36:16.783 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:36:16.783 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:36:16.783 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:36:16.783 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:17.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:36:17.009 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:18.011 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:36:18.011 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:18.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:36:18.239 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:19.240 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:36:19.240 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:19.470 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:36:19.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:36:19.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:19.471 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:36:19.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:36:20.025 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:36:20.026 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:36:20.026 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:20.026 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:20.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:36:20.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:36:20.349 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:36:20.350 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1307: test_objectstore_tool: rados --pool rbd put GROUP /etc/group 2026-02-13T19:36:20.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1308: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 GROUP get-bytes 2026-02-13T19:36:20.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 GROUP get-bytes 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1309: test_objectstore_tool: diff - /etc/group 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-13T19:36:20.370 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:20.371 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:20.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:20.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:20.371 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:20.474 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:20.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 GROUP get-bytes 2026-02-13T19:36:20.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-13T19:36:20.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:36:20.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-13T19:36:20.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:36:20.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-13T19:36:20.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 GROUP get-bytes 2026-02-13T19:36:20.573 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/ceph-helpers 0 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:36:20.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:20.575 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-13T19:36:20.576 INFO:tasks.workunit.client.0.trial162.stderr:start osd.0 2026-02-13T19:36:20.577 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:36:20.577 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-13T19:36:20.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-13T19:36:20.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:36:20.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:36:20.578 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:36:20.588 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:20.587+0000 7f4426a2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:20.589 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:20.588+0000 7f4426a2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:20.590 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:20.589+0000 7f4426a2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:20.716 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:20.714+0000 7f4426a2e780 -1 Falling back to public interface 2026-02-13T19:36:20.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-13T19:36:20.808 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:0 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:20.809 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:21.011 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:21.010+0000 7f4426a2e780 -1 osd.0 12 log_to_monitors true 2026-02-13T19:36:21.039 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:36:22.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:36:22.041 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:22.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:36:22.042 INFO:tasks.workunit.client.0.trial162.stderr:1 2026-02-13T19:36:22.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:22.042 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:osd.0 up in weight 1 up_from 15 up_thru 15 down_at 13 last_clean_interval [5,12) [v2:127.0.0.1:6800/1465505540,v1:127.0.0.1:6801/1465505540] [v2:127.0.0.1:6802/1465505540,v1:127.0.0.1:6803/1465505540] exists,up 853e72cf-5304-4725-93d8-03c98765817b 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:36:22.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:36:22.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:36:22.275 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:36:22.275 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:36:22.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:36:22.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:36:22.275 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:36:22.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:36:22.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:36:22.356 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:36:22.357 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:36:22.357 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:36:22.357 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:36:22.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:36:22.588 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:36:22.589 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:36:22.589 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:36:22.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-13T19:36:22.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-13T19:36:22.654 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-64424509443' 2026-02-13T19:36:22.655 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:36:22.655 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-64424509443 2026-02-13T19:36:22.655 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:36:22.655 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:36:22.656 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-64424509443 2026-02-13T19:36:22.656 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:36:22.656 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509443 2026-02-13T19:36:22.657 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 64424509443' 2026-02-13T19:36:22.657 INFO:tasks.workunit.client.0.trial162.stderr:waiting osd.0 seq 64424509443 2026-02-13T19:36:22.657 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:22.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-13T19:36:22.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:36:22.895 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:22.895 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:23.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:36:23.212 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:36:23.212 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:36:23.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:36:23.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:36:23.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:36:23.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:36:23.213 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:36:23.438 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:36:23.438 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:36:23.439 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:23.439 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1310: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:36:23.762 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:23.763 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:23.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:23.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-13T19:36:23.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-13T19:36:23.867 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:36:23.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-13T19:36:23.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:36:23.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-13T19:36:23.868 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 NOTEXISTS get-bytes 2026-02-13T19:36:23.913 INFO:tasks.workunit.client.0.trial162.stderr:No object id 'NOTEXISTS' found or invalid JSON specified 2026-02-13T19:36:23.956 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-13T19:36:23.956 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: return 1 2026-02-13T19:36:23.956 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: return 1 2026-02-13T19:36:23.956 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1311: test_objectstore_tool: teardown td/ceph-helpers 2026-02-13T19:36:23.956 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:23.956 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:23.957 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:23.957 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:23.957 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:23.957 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:23.957 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:23.958 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:24.060 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:24.061 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:24.061 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:24.062 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:24.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:24.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:24.064 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:24.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:24.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:24.065 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:24.066 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:24.066 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:24.067 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:24.067 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:24.071 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:24.071 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.071 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:24.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:24.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:24.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_schedule_scrub td/ceph-helpers 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1987: test_pg_schedule_scrub: local dir=td/ceph-helpers 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1989: test_pg_schedule_scrub: setup td/ceph-helpers 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:24.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:24.075 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:24.075 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:24.075 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:24.076 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:24.076 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:24.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:24.077 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:24.077 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:24.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:24.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:24.078 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:24.079 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:24.079 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:24.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:24.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:24.081 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:24.082 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.082 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:24.082 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:24.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:24.083 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:36:24.084 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:36:24.084 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.084 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.084 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:36:24.085 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:36:24.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:36:24.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:36:24.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:36:24.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1990: test_pg_schedule_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:24.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:36:24.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:36:24.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:36:24.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:36:24.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:36:24.087 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:24.104 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:36:24.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:24.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:24.105 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:24.105 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.105 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.105 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:24.105 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:24.129 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:36:24.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:36:24.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:24.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:24.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:36:24.131 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:24.132 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:36:24.175 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:36:24.175 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:24.175 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:24.175 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:36:24.175 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:36:24.175 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:24.175 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:24.176 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:24.176 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:24.176 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.176 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.176 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:24.176 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:24.176 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:36:24.216 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1991: test_pg_schedule_scrub: run_mgr td/ceph-helpers x --mgr_stats_period=1 2026-02-13T19:36:24.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:36:24.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:36:24.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:36:24.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:36:24.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:36:24.217 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:36:24.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:36:24.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:24.321 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:24.321 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:24.322 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.322 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.322 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:24.322 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:36:24.323 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1992: test_pg_schedule_scrub: run_osd td/ceph-helpers 0 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:36:24.339 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:36:24.340 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:24.340 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:24.340 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:24.340 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:24.340 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:24.340 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:24.340 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e06ed78-9d5c-497d-b3a7-66a93ed59916 2026-02-13T19:36:24.341 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 4e06ed78-9d5c-497d-b3a7-66a93ed59916 2026-02-13T19:36:24.342 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4e06ed78-9d5c-497d-b3a7-66a93ed59916' 2026-02-13T19:36:24.342 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:36:24.352 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA4fY9p6Ov0FBAAf1A1IvSKohXPVNHeHtAzoQ== 2026-02-13T19:36:24.352 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA4fY9p6Ov0FBAAf1A1IvSKohXPVNHeHtAzoQ=="}' 2026-02-13T19:36:24.352 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e06ed78-9d5c-497d-b3a7-66a93ed59916 -i td/ceph-helpers/0/new.json 2026-02-13T19:36:24.457 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:24.464 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:36:24.465 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA4fY9p6Ov0FBAAf1A1IvSKohXPVNHeHtAzoQ== --osd-uuid 4e06ed78-9d5c-497d-b3a7-66a93ed59916 2026-02-13T19:36:24.482 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:24.481+0000 7fc22c839780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:24.484 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:24.483+0000 7fc22c839780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:24.485 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:24.484+0000 7fc22c839780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:24.485 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:24.484+0000 7fc22c839780 -1 bdev(0x4130c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:36:24.485 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:24.484+0000 7fc22c839780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:36:24.944 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:36:24.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:36:24.945 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:36:24.946 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:36:24.946 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:36:25.049 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:36:25.050 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:36:25.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:36:25.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:36:25.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:36:25.051 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:36:25.064 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:25.063+0000 7f8de9439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:25.066 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:25.065+0000 7f8de9439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:25.067 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:25.066+0000 7f8de9439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:25.150 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:36:25.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:36:25.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:36:25.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:36:25.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:36:25.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:25.151 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:36:25.151 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:25.152 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:25.152 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:25.248 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:36:25.267 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:25.266+0000 7f8de9439780 -1 Falling back to public interface 2026-02-13T19:36:25.647 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:25.645+0000 7f8de9439780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:36:26.249 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:36:26.250 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:26.250 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:36:26.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:36:26.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:26.251 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:26.354 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2627317458,v1:127.0.0.1:6801/2627317458] [v2:127.0.0.1:6802/2627317458,v1:127.0.0.1:6803/2627317458] exists,up 4e06ed78-9d5c-497d-b3a7-66a93ed59916 2026-02-13T19:36:26.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:36:26.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:36:26.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:36:26.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1993: test_pg_schedule_scrub: create_rbd_pool 2026-02-13T19:36:26.355 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:36:26.450 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:36:26.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:36:26.455 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:36:26.709 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:36:26.716 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:36:27.718 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:36:27.997 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1994: test_pg_schedule_scrub: wait_for_clean 2026-02-13T19:36:27.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:36:27.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:36:27.998 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:36:27.998 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:36:27.998 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:36:27.998 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:36:27.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:36:27.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:36:27.999 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:36:28.080 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:36:28.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:36:28.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:36:28.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:36:28.089 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:36:28.089 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:36:28.311 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:36:28.311 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:36:28.311 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:36:28.312 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:36:28.377 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T19:36:28.378 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T19:36:28.378 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T19:36:28.378 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:36:28.378 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-13T19:36:28.379 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:36:28.379 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:36:28.380 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T19:36:28.380 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:36:28.381 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-13T19:36:28.381 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836484 2026-02-13T19:36:28.382 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T19:36:28.382 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:28.612 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T19:36:28.613 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:29.613 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:36:29.614 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:29.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T19:36:29.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:30.842 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:36:30.843 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:31.072 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836484 2026-02-13T19:36:31.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:36:31.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:31.073 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:31.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:36:31.394 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:36:31.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:36:31.394 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:36:31.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:36:31.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:36:31.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:36:31.395 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:36:31.620 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:36:31.620 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:36:31.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:31.621 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1995: test_pg_schedule_scrub: pg_schedule_scrub 1.0 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1969: pg_schedule_scrub: local pgid=1.0 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: pg_schedule_scrub: wait_for_pg_clean 1.0 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-13T19:36:31.936 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-13T19:36:31.937 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:36:31.937 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:36:31.937 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:36:31.937 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:36:31.937 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:36:32.137 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-13T19:36:32.138 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-13T19:36:32.138 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-13T19:36:32.138 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-13T19:36:32.138 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:36:32.138 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:36:32.363 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:36:32.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:36:32.364 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:36:32.364 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:36:32.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-13T19:36:32.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-13T19:36:32.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-13T19:36:32.430 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:36:32.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-13T19:36:32.431 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:36:32.431 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:36:32.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-13T19:36:32.432 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:36:32.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-13T19:36:32.433 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-13T19:36:32.433 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836490 2026-02-13T19:36:32.433 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:32.664 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T19:36:32.665 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:33.665 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:36:33.666 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:33.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-13T19:36:33.892 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:34.893 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:36:34.894 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:35.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836492 -lt 21474836490 2026-02-13T19:36:35.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T19:36:35.121 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-13T19:36:35.122 INFO:tasks.workunit.client.0.trial162.stdout:#---------- 1.0 loop 0 2026-02-13T19:36:35.122 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T19:36:35.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T19:36:35.123 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T19:36:35.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T19:36:35.123 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: get_last_scrub_stamp 1.0 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:35.190 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:35.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: local last_scrub=2026-02-13T19:36:26.644398+0000 2026-02-13T19:36:35.417 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: pg_schedule_scrub: ceph tell 1.0 schedule-scrub 2026-02-13T19:36:35.481 INFO:tasks.workunit.client.0.trial162.stdout:{ 2026-02-13T19:36:35.482 INFO:tasks.workunit.client.0.trial162.stdout: "deep": false, 2026-02-13T19:36:35.482 INFO:tasks.workunit.client.0.trial162.stdout: "must": false, 2026-02-13T19:36:35.482 INFO:tasks.workunit.client.0.trial162.stdout: "stamp": "2026-02-11T07:36:35.482106+0000" 2026-02-13T19:36:35.482 INFO:tasks.workunit.client.0.trial162.stdout:} 2026-02-13T19:36:35.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: pg_schedule_scrub: wait_for_scrub 1.0 2026-02-13T19:36:26.644398+0000 2026-02-13T19:36:35.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T19:36:35.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T19:36:26.644398+0000 2026-02-13T19:36:35.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T19:36:35.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T19:36:35.487 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:35.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:35.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:35.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:35.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:35.488 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:35.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:26.644398+0000 '>' 2026-02-13T19:36:26.644398+0000 2026-02-13T19:36:35.713 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T19:36:36.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T19:36:36.714 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:36.715 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:36.715 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:36.715 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:36.715 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:36.715 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:36.939 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:26.644398+0000 '>' 2026-02-13T19:36:26.644398+0000 2026-02-13T19:36:36.939 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T19:36:37.940 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T19:36:37.940 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:37.940 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:37.940 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:37.940 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:37.940 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:37.941 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:38.169 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:26.644398+0000 '>' 2026-02-13T19:36:26.644398+0000 2026-02-13T19:36:38.169 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T19:36:39.170 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T19:36:39.171 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:39.171 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:39.171 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:39.171 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:39.172 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:39.172 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:35.526804+0000 '>' 2026-02-13T19:36:26.644398+0000 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1996: test_pg_schedule_scrub: teardown td/ceph-helpers 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:39.403 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:39.510 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:39.511 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:39.511 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:39.512 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:39.513 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:39.513 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:39.514 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:39.514 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:39.515 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:39.515 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:39.515 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:39.516 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:39.517 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:39.517 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:39.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:39.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.522 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:39.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:39.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:39.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-13T19:36:39.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_scrub td/ceph-helpers 2026-02-13T19:36:39.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: test_pg_scrub: local dir=td/ceph-helpers 2026-02-13T19:36:39.524 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: test_pg_scrub: setup td/ceph-helpers 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:39.525 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:39.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:39.526 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:36:39.526 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:36:39.527 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:36:39.527 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:36:39.527 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:36:39.528 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:36:39.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:39.528 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:36:39.529 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:36:39.529 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:36:39.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:36:39.530 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:36:39.531 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T19:36:39.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:36:39.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.532 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.532 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T19:36:39.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:36:39.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:36:39.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-13T19:36:39.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:36:39.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.534 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76588 2026-02-13T19:36:39.536 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:36:39.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:36:39.536 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: test_pg_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-13T19:36:39.537 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:39.555 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:36:39.555 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:39.555 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:39.555 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:39.555 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.556 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.556 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:39.556 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-13T19:36:39.582 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:36:39.584 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:36:39.584 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:39.584 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:39.584 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:36:39.584 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:36:39.584 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:39.584 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:39.585 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:39.585 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:39.585 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.585 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.585 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:39.585 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:39.585 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get fsid 2026-02-13T19:36:39.627 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:36:39.627 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:36:39.627 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76588/ceph-mon.a.asok 2026-02-13T19:36:39.628 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:36:39.629 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76588/ceph-mon.a.asok config get mon_host 2026-02-13T19:36:39.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: test_pg_scrub: run_mgr td/ceph-helpers x 2026-02-13T19:36:39.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-13T19:36:39.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:36:39.670 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:36:39.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:36:39.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-13T19:36:39.671 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:39.770 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:36:39.771 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:36:39.789 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: test_pg_scrub: run_osd td/ceph-helpers 0 2026-02-13T19:36:39.789 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-13T19:36:39.790 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:39.791 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:36:39.792 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:36:39.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:36:39.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-13T19:36:39.793 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:36:39.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e2b29c96-a9ec-4425-8fda-9db6e192077e 2026-02-13T19:36:39.793 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e2b29c96-a9ec-4425-8fda-9db6e192077e' 2026-02-13T19:36:39.793 INFO:tasks.workunit.client.0.trial162.stdout:add osd0 e2b29c96-a9ec-4425-8fda-9db6e192077e 2026-02-13T19:36:39.794 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:36:39.801 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBHfY9pWXesLxAApV3+V7jUdFKba/tllRrkig== 2026-02-13T19:36:39.801 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBHfY9pWXesLxAApV3+V7jUdFKba/tllRrkig=="}' 2026-02-13T19:36:39.801 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e2b29c96-a9ec-4425-8fda-9db6e192077e -i td/ceph-helpers/0/new.json 2026-02-13T19:36:39.903 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:39.909 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-13T19:36:39.910 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBHfY9pWXesLxAApV3+V7jUdFKba/tllRrkig== --osd-uuid e2b29c96-a9ec-4425-8fda-9db6e192077e 2026-02-13T19:36:39.926 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:39.925+0000 7fe20b831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:39.928 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:39.927+0000 7fe20b831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:39.929 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:39.927+0000 7fe20b831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:39.929 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:39.928+0000 7fe20b831780 -1 bdev(0x3fa6c00 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:36:39.929 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:39.928+0000 7fe20b831780 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-13T19:36:40.368 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-13T19:36:40.368 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:36:40.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:36:40.369 INFO:tasks.workunit.client.0.trial162.stdout:adding osd0 key to auth repository 2026-02-13T19:36:40.369 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:36:40.474 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:36:40.474 INFO:tasks.workunit.client.0.trial162.stdout:start osd.0 2026-02-13T19:36:40.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=cc6170d8-16c6-4168-afc1-a587f537dbfd --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.76588/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:36:40.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:36:40.475 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:36:40.476 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:36:40.489 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:40.488+0000 7fa19a844780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:40.490 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:40.489+0000 7fa19a844780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:40.491 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:40.490+0000 7fa19a844780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:36:40.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:36:40.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:36:40.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:36:40.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:36:40.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:36:40.574 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:40.575 INFO:tasks.workunit.client.0.trial162.stdout:0 2026-02-13T19:36:40.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:36:40.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:40.575 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:40.656 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:40.655+0000 7fa19a844780 -1 Falling back to public interface 2026-02-13T19:36:40.672 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:36:41.053 INFO:tasks.workunit.client.0.trial162.stderr:2026-02-13T19:36:41.052+0000 7fa19a844780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:36:41.673 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:36:41.674 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:36:41.674 INFO:tasks.workunit.client.0.trial162.stdout:1 2026-02-13T19:36:41.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:36:41.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:36:41.675 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:36:41.774 INFO:tasks.workunit.client.0.trial162.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1394049828,v1:127.0.0.1:6801/1394049828] [v2:127.0.0.1:6802/1394049828,v1:127.0.0.1:6803/1394049828] exists,up e2b29c96-a9ec-4425-8fda-9db6e192077e 2026-02-13T19:36:41.774 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:36:41.774 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:36:41.775 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:36:41.775 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1947: test_pg_scrub: create_rbd_pool 2026-02-13T19:36:41.775 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:36:41.870 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' does not exist 2026-02-13T19:36:41.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:36:41.876 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:36:42.122 INFO:tasks.workunit.client.0.trial162.stderr:pool 'rbd' already exists 2026-02-13T19:36:42.130 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:36:43.131 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:36:43.411 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1948: test_pg_scrub: wait_for_clean 2026-02-13T19:36:43.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:36:43.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:36:43.412 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:36:43.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:36:43.413 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:36:43.413 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:36:43.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:36:43.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:36:43.413 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:36:43.497 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:36:43.497 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:36:43.497 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:36:43.497 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:36:43.498 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:36:43.498 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:36:43.728 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:36:43.728 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:36:43.728 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:36:43.729 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:36:43.794 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-13T19:36:43.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-13T19:36:43.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-13T19:36:43.795 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:36:43.795 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-13T19:36:43.795 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:36:43.797 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:36:43.797 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-13T19:36:43.797 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:36:43.798 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-13T19:36:43.798 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-13T19:36:43.798 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836483 2026-02-13T19:36:43.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:44.026 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:36:44.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:45.027 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:36:45.028 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:45.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-13T19:36:45.254 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:46.255 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:36:46.255 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:46.482 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-13T19:36:46.482 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:36:46.483 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:46.483 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:46.798 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:36:46.798 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:36:46.798 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:36:46.798 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:36:46.798 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:36:46.798 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:36:46.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:36:46.799 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:36:47.022 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:36:47.023 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:36:47.023 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:36:47.023 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1949: test_pg_scrub: pg_scrub 1.0 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-13T19:36:47.341 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:36:47.342 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:36:47.342 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:36:47.342 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:36:47.342 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:36:47.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-13T19:36:47.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-13T19:36:47.540 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-13T19:36:47.541 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-13T19:36:47.541 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:36:47.541 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:36:47.775 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:36:47.775 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:36:47.775 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:36:47.775 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:36:47.840 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-13T19:36:47.840 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-13T19:36:47.840 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836486' 2026-02-13T19:36:47.840 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:36:47.840 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836486 2026-02-13T19:36:47.841 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:36:47.841 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:36:47.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-13T19:36:47.842 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:36:47.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836486 2026-02-13T19:36:47.843 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-13T19:36:47.843 INFO:tasks.workunit.client.0.trial162.stdout:waiting osd.0 seq 21474836486 2026-02-13T19:36:47.844 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:48.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-13T19:36:48.073 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:49.074 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:36:49.074 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:49.303 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-13T19:36:49.303 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:36:50.304 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:36:50.304 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:36:50.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-13T19:36:50.533 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T19:36:50.534 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-13T19:36:50.534 INFO:tasks.workunit.client.0.trial162.stdout:#---------- 1.0 loop 0 2026-02-13T19:36:50.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T19:36:50.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T19:36:50.535 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T19:36:50.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T19:36:50.535 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T19:36:50.600 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:50.601 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:50.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-13T19:36:42.057846+0000 2026-02-13T19:36:50.833 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-13T19:36:51.054 INFO:tasks.workunit.client.0.trial162.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-13T19:36:51.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-13T19:36:42.057846+0000 2026-02-13T19:36:51.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-13T19:36:51.062 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-13T19:36:42.057846+0000 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:51.063 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:51.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:42.057846+0000 '>' 2026-02-13T19:36:42.057846+0000 2026-02-13T19:36:51.291 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T19:36:52.292 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T19:36:52.292 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:52.293 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:52.293 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:52.293 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:52.293 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:52.293 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:52.520 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:42.057846+0000 '>' 2026-02-13T19:36:42.057846+0000 2026-02-13T19:36:52.520 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T19:36:53.521 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T19:36:53.521 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:53.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:53.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:53.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:53.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:53.522 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:53.750 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:42.057846+0000 '>' 2026-02-13T19:36:42.057846+0000 2026-02-13T19:36:53.750 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-13T19:36:54.751 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-13T19:36:54.752 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-13T19:36:54.752 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-13T19:36:54.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-13T19:36:54.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-13T19:36:54.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-13T19:36:54.753 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-13T19:36:54.978 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-13T19:36:51.101854+0000 '>' 2026-02-13T19:36:42.057846+0000 2026-02-13T19:36:54.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-13T19:36:54.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1950: test_pg_scrub: kill_daemons td/ceph-helpers KILL osd 2026-02-13T19:36:54.979 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:36:54.979 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:36:54.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:36:54.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:36:54.979 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:36:55.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:36:55.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: WAIT_FOR_CLEAN_TIMEOUT=10 2026-02-13T19:36:55.085 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: TIMEOUT=2 2026-02-13T19:36:55.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: pg_scrub 1.0 2026-02-13T19:36:55.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-13T19:36:55.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-13T19:36:55.086 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-13T19:36:55.086 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 10 1 3 2026-02-13T19:36:55.087 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:36:55.087 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:36:55.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:36:55.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:36:55.087 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:36:55.119 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '1') 2026-02-13T19:36:55.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-13T19:36:55.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-13T19:36:55.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-13T19:36:55.120 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=2 2026-02-13T19:36:55.120 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:36:55.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-13T19:36:55.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:36:55.354 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:36:55.354 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 2 ceph tell osd.0 flush_pg_stats 2026-02-13T19:36:57.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-13T19:36:57.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-13T19:36:57.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-13T19:36:57.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-13T19:36:57.360 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T19:36:57.361 INFO:tasks.workunit.client.0.trial162.stdout:#---------- 1.0 loop 0 2026-02-13T19:36:57.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-13T19:36:57.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T19:36:57.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T19:36:57.362 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T19:36:57.362 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T19:36:57.362 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T22:33:57.150 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1: is_pg_clean: teardown td/ceph-helpers 1 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=1 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T22:33:57.177 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T22:33:57.260 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T22:33:57.260 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T22:33:57.261 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T22:33:57.262 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T22:33:57.263 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T22:33:57.264 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T22:33:57.264 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T22:33:57.265 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T22:33:57.266 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T22:33:57.266 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T22:33:57.266 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T22:33:57.267 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T22:33:57.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-13T22:33:57.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:181: teardown: '[' -n '' ']' 2026-02-13T22:33:57.268 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:185: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-13T22:33:57.269 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:186: teardown: mv td/ceph-helpers/mgr.x.log td/ceph-helpers/mon.a.log td/ceph-helpers/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-13T22:33:57.269 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-13T22:33:57.273 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T22:33:57.273 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T22:33:57.273 INFO:tasks.workunit.client.0.trial162.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76588 2026-02-13T22:33:57.273 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76588 2026-02-13T22:33:57.274 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T22:33:57.274 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T22:33:57.274 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-13T22:33:57.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 0 >= 5 )) 2026-02-13T22:33:57.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 1 2026-02-13T22:33:58.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-13T22:33:58.275 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-13T22:33:58.276 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 1' 2026-02-13T22:33:58.276 INFO:tasks.workunit.client.0.trial162.stdout:#---------- 1.0 loop 1 2026-02-13T22:33:58.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-13T22:33:58.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-13T22:33:58.277 INFO:tasks.workunit.client.0.trial162.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-13T22:33:58.277 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-13T22:33:58.277 INFO:tasks.workunit.client.0.trial162.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-13T22:34:57.177 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-13T22:35:57.179 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-13T22:35:57.179 INFO:tasks.workunit:Stopping ['misc'] on client.0... 2026-02-13T22:35:57.179 DEBUG:teuthology.orchestra.run.trial162:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-13T22:35:57.516 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/remote.py", line 575, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test misc/test-ceph-helpers.sh) on trial162 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh' 2026-02-13T22:35:57.521 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-13T22:35:57.537 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-13T22:35:57.537 DEBUG:teuthology.orchestra.run.trial162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T22:35:57.667 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-13T22:35:57.679 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-13T22:35:57.679 DEBUG:teuthology.orchestra.run.trial162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T22:35:57.695 INFO:teuthology.orchestra.run.trial162.stderr:bash: line 1: ntpq: command not found 2026-02-13T22:35:57.724 INFO:teuthology.orchestra.run.trial162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T22:35:57.724 INFO:teuthology.orchestra.run.trial162.stdout:=============================================================================== 2026-02-13T22:35:57.724 INFO:teuthology.orchestra.run.trial162.stdout:^- sensei.ruselabs.com 2 10 377 767 -966us[ -966us] +/- 38ms 2026-02-13T22:35:57.724 INFO:teuthology.orchestra.run.trial162.stdout:^* core-anycast.ntp00.hosti> 2 10 377 1023 +383us[ +490us] +/- 28ms 2026-02-13T22:35:57.724 INFO:teuthology.orchestra.run.trial162.stdout:^- 172-233-177-198.ip.linod> 2 10 377 49 -5480us[-5480us] +/- 84ms 2026-02-13T22:35:57.724 INFO:teuthology.orchestra.run.trial162.stdout:^- us-east-2.clearnet.pw 2 9 377 226 -2728us[-2728us] +/- 54ms 2026-02-13T22:35:57.725 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-13T22:35:57.737 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-13T22:35:57.737 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-13T22:35:57.753 DEBUG:teuthology.orchestra.run.trial162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T22:35:57.818 DEBUG:teuthology.orchestra.run.trial162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-13T22:35:57.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T22:35:57.845 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-13T22:35:57.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-13T22:35:57.875 INFO:teuthology.task.internal:Duration was 11596.092709 seconds 2026-02-13T22:35:57.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-13T22:35:57.890 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-13T22:35:57.890 DEBUG:teuthology.orchestra.run.trial162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T22:35:57.927 INFO:teuthology.orchestra.run.trial162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T22:35:58.273 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-13T22:35:58.273 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial162.front.sepia.ceph.com 2026-02-13T22:35:58.273 DEBUG:teuthology.orchestra.run.trial162:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T22:35:58.331 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-13T22:35:58.332 DEBUG:teuthology.orchestra.run.trial162:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T22:35:58.435 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-13T22:35:58.436 DEBUG:teuthology.orchestra.run.trial162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T22:35:58.461 INFO:teuthology.orchestra.run.trial162.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T22:35:58.461 INFO:teuthology.orchestra.run.trial162.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T22:35:58.461 INFO:teuthology.orchestra.run.trial162.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T22:35:58.461 INFO:teuthology.orchestra.run.trial162.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-13T22:35:58.461 INFO:teuthology.orchestra.run.trial162.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T22:35:58.471 INFO:teuthology.orchestra.run.trial162.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T22:35:58.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-13T22:35:58.491 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-13T22:35:58.491 DEBUG:teuthology.orchestra.run.trial162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T22:35:58.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-13T22:35:58.548 DEBUG:teuthology.orchestra.run.trial162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T22:35:58.600 INFO:teuthology.orchestra.run.trial162.stdout:kernel.core_pattern = core 2026-02-13T22:35:58.611 DEBUG:teuthology.orchestra.run.trial162:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T22:35:58.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T22:35:58.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-13T22:35:58.679 INFO:teuthology.task.internal:Transferring archived files... 2026-02-13T22:35:58.681 DEBUG:teuthology.misc:Transferring archived files from trial162:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49813/remote/trial162 2026-02-13T22:35:58.682 DEBUG:teuthology.orchestra.run.trial162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T22:36:12.328 INFO:teuthology.task.internal:Removing archive directory... 2026-02-13T22:36:12.328 DEBUG:teuthology.orchestra.run.trial162:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T22:36:12.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-13T22:36:12.430 INFO:teuthology.task.internal:Not uploading archives. 2026-02-13T22:36:12.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-13T22:36:12.458 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-13T22:36:12.458 DEBUG:teuthology.orchestra.run.trial162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T22:36:12.476 INFO:teuthology.orchestra.run.trial162.stdout: 786525 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 13 22:36 /home/ubuntu/cephtest 2026-02-13T22:36:12.476 INFO:teuthology.orchestra.run.trial162.stdout: 1056496 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 13 19:27 /home/ubuntu/cephtest/mnt.0 2026-02-13T22:36:12.476 INFO:teuthology.orchestra.run.trial162.stdout: 1056497 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 13 19:33 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-13T22:36:12.476 INFO:teuthology.orchestra.run.trial162.stdout: 1056540 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 13 19:33 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T22:36:12.476 INFO:teuthology.orchestra.run.trial162.stdout: 1072391 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 22:33 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-13T22:36:12.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T22:36:12.477 INFO:teuthology.orchestra.run.trial162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-13T22:36:12.477 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial162 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-13T22:36:12.479 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-13T22:36:12.494 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-13T22:36:12.510 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial162 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-13T22:36:12.516 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/misc} duration: 11596.092709064484 failure_reason: 'Command failed (workunit test misc/test-ceph-helpers.sh) on trial162 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-13T22:36:12.516 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T22:36:12.633 INFO:teuthology.run:FAIL